dataframe 根据日期重采样 计算个数

量化交易李魔佛 发表了文章 • 0 个评论 • 125 次浏览 • 2019-12-19 09:07 • 来自相关话题

按照日期重新采样,计算每天的个数new_df = df.resample('D').count()
按照日期重新采样,计算每天的个数new_df = df.resample('D').count()

失落

闲聊李魔佛 发表了文章 • 0 个评论 • 113 次浏览 • 2019-12-17 23:46 • 来自相关话题

记住今天,铭记。
记住今天,铭记。

bandcamp移动开发更简单

数据库linxiaojue 发表了文章 • 0 个评论 • 115 次浏览 • 2019-12-14 05:12 • 来自相关话题

bandcamp移动开发更简单http://ydkfpgjd.bandcamp.com/
http://TalkingData.bandcamp.com/
http://Bugly.bandcamp.com/
http://Box2D.bandcamp.com/
http://aineice.bandcamp.com/
http://wyyp.bandcamp.com/
http://Prepo.bandcamp.com/
http://Chipmunk.bandcamp.com/
http://openinstall.bandcamp.com/
http://MobileInsight.bandcamp.com/
http://zhugelo.bandcamp.com/
http://CobubRazor.bandcamp.com/
http://Testin.bandcamp.com/
http://crashlytics.bandcamp.com/
http://APKProtect.bandcamp.com/
http://Ucloud.bandcamp.com/
http://ydkfpgj.bandcamp.com/releases
http://TalkingData.bandcamp.com/releases
http://Bugly.bandcamp.com/releases
http://Box2D.bandcamp.com/releases
http://aineice.bandcamp.com/releases
http://wyyp.bandcamp.com/releases
http://Prepo.bandcamp.com/releases
http://Chipmunk.bandcamp.com/releases
http://openinstall.bandcamp.com/releases
http://MobileInsight.bandcamp.com/releases
http://zhugelo.bandcamp.com/releases
http://CobubRazor.bandcamp.com/releases
http://Testin.bandcamp.com/releases
http://crashlytics.bandcamp.com/releases
http://APKProtect.bandcamp.com/releases
http://Ucloud.bandcamp.com/releases 查看全部

mongodb日期条件查找

数据库李魔佛 发表了文章 • 0 个评论 • 125 次浏览 • 2019-12-10 10:41 • 来自相关话题

有时候用new Date() 不管用
有时候用new Date() 不管用

多线程调用yolo模型会出错(包括使用cv2载入yolo模型)

深度学习李魔佛 发表了文章 • 0 个评论 • 136 次浏览 • 2019-12-05 10:27 • 来自相关话题

占坑。
 
多线程调用yolo模型会出错(包括使用cv2载入yolo模型)
 
占坑。
 
多线程调用yolo模型会出错(包括使用cv2载入yolo模型)
 

keras yolo物体检测 入门教程

深度学习李魔佛 发表了文章 • 0 个评论 • 220 次浏览 • 2019-11-28 16:03 • 来自相关话题

占坑
占坑

RuntimeError: `get_session` is not available when using TensorFlow 2.0.

深度学习李魔佛 发表了文章 • 0 个评论 • 1320 次浏览 • 2019-11-28 15:10 • 来自相关话题

这个问题是TensorFlow版本问题,在2.0以上get_session是被移除了。需要做一些修改,或者把tf降级。可以安装1.15版本
pip install tensorflow==1.15 --upgradeHere, we will see how we can upgrade our code to work with tensorflow 2.0.

This error is usually faced when we are loading pre-trained model with tensorflow session/graph or we are building flask api over a pre-trained model and loading model in tensorflow graph to avoid collision of sessions while application is getting multiple requests at once or say in case of multi-threadinng

After tensorflow 2.0 upgrade, i also started facing above error in one of my project when i had built api of pre-trained model with flask. So i looked around in tensorflow 2.0 documents to find a workaround, to avoid this runtime error and upgrade my code to work with tensorflow 2.0 as well rather than downgrading it to tensorflow 1.x .

I had a project on which i had written tutorial as well on how to build Flask api on trained keras model of text classification and then use it in production

But this project was not working after tensorflow upgrade and was facing runtime error.

Stacktrace of error was something like below:

File "/Users/Upasana/Documents/playground/deploy-keras-model-in-production/src/main.py", line 37, in model_predict
with backend.get_session().graph.as_default() as g:
File "/Users/Upasana/Documents/playground/deploy-keras-model-in-production/venv-tf2/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 379, in get_session
'`get_session` is not available '
RuntimeError: `get_session` is not available when using TensorFlow 2.0.
Related code to get model
with backend.get_session().graph.as_default() as g:
model = SentimentService.get_model1()
Related code to load model
def load_deep_model(self, model):
json_file = open('./src/mood-saved-models/' + model + '.json', 'r')
loaded_model_json = json_file.read()
loaded_model = model_from_json(loaded_model_json)

loaded_model.load_weights("./src/mood-saved-models/" + model + ".h5")

loaded_model._make_predict_function()
return loaded_model
get_session is removed in tensorflow 2.0 and hence not available.

so, in order to load saved model we switched methods. Rather than using keras’s load_model, we used tensorflow to load model so that we can load model using distribution strategy.

Note
The tf.distribute.Strategy API provides an abstraction for distributing your training across multiple processing units.

New code to get model
another_strategy = tf.distribute.MirroredStrategy()
with another_strategy.scope():
model = SentimentService.get_model1()
New code to load model
def load_deep_model(self, model):
loaded_model = tf.keras.models.load_model("./src/mood-saved-models/"model + ".h5")
return loaded_model
This worked and solved the problem with runtime error of get_session not available in tensorflow 2.0 . You can refer to Tensorflow 2.0 upgraded article too

Hope, this will solve your problem too. Thanks for following this article. 查看全部
这个问题是TensorFlow版本问题,在2.0以上get_session是被移除了。需要做一些修改,或者把tf降级。可以安装1.15版本
pip install tensorflow==1.15 --upgrade
Here, we will see how we can upgrade our code to work with tensorflow 2.0.

This error is usually faced when we are loading pre-trained model with tensorflow session/graph or we are building flask api over a pre-trained model and loading model in tensorflow graph to avoid collision of sessions while application is getting multiple requests at once or say in case of multi-threadinng

After tensorflow 2.0 upgrade, i also started facing above error in one of my project when i had built api of pre-trained model with flask. So i looked around in tensorflow 2.0 documents to find a workaround, to avoid this runtime error and upgrade my code to work with tensorflow 2.0 as well rather than downgrading it to tensorflow 1.x .

I had a project on which i had written tutorial as well on how to build Flask api on trained keras model of text classification and then use it in production

But this project was not working after tensorflow upgrade and was facing runtime error.

Stacktrace of error was something like below:

File "/Users/Upasana/Documents/playground/deploy-keras-model-in-production/src/main.py", line 37, in model_predict
with backend.get_session().graph.as_default() as g:
File "/Users/Upasana/Documents/playground/deploy-keras-model-in-production/venv-tf2/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 379, in get_session
'`get_session` is not available '
RuntimeError: `get_session` is not available when using TensorFlow 2.0.
Related code to get model
with backend.get_session().graph.as_default() as g:
model = SentimentService.get_model1()
Related code to load model
def load_deep_model(self, model):
json_file = open('./src/mood-saved-models/' + model + '.json', 'r')
loaded_model_json = json_file.read()
loaded_model = model_from_json(loaded_model_json)

loaded_model.load_weights("./src/mood-saved-models/" + model + ".h5")

loaded_model._make_predict_function()
return loaded_model
get_session is removed in tensorflow 2.0 and hence not available.

so, in order to load saved model we switched methods. Rather than using keras’s load_model, we used tensorflow to load model so that we can load model using distribution strategy.

Note
The tf.distribute.Strategy API provides an abstraction for distributing your training across multiple processing units.

New code to get model
another_strategy = tf.distribute.MirroredStrategy()
with another_strategy.scope():
model = SentimentService.get_model1()
New code to load model
def load_deep_model(self, model):
loaded_model = tf.keras.models.load_model("./src/mood-saved-models/"model + ".h5")
return loaded_model
This worked and solved the problem with runtime error of get_session not available in tensorflow 2.0 . You can refer to Tensorflow 2.0 upgraded article too

Hope, this will solve your problem too. Thanks for following this article.

yolo voc_label 源码分析

深度学习李魔佛 发表了文章 • 0 个评论 • 137 次浏览 • 2019-11-27 15:19 • 来自相关话题

作用读取每个xml文件,把坐标转化为相对坐标,对应文件名保存起来
 
import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import join




sets = [('2012', 'train'), ('2012', 'val'), ('2007', 'train'), ('2007', 'val'), ('2007', 'test')]

# 20类
classes = ["aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog",
"horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"]

# size w,h
# box x-min,x-max,y-min,y-max
def convert(size, box):
dw = 1. / size[0]
dh = 1. / size[1]
x = (box[0] + box[1]) / 2.0 # 中心点位置
y = (box[2] + box[3]) / 2.0
w = box[1] - box[0]
h = box[3] - box[2]
x = x * dw
w = w * dw
y = y * dh
h = h * dh # 全部转化为相对坐标
return (x, y, w, h)


def convert_annotation(year, image_id):
# 找到2个同样的文件
in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml' % (year, image_id))
out_file = open('VOCdevkit/VOC%s/labels/%s.txt' % (year, image_id), 'w')

tree = ET.parse(in_file)
root = tree.getroot()
size = root.find('size')
w = int(size.find('width').text)
h = int(size.find('height').text)

for obj in root.iter('object'):
difficult = obj.find('difficult').text
cls = obj.find('name').text
if cls not in classes or int(difficult) == 1: # difficult ==1 的不要了
continue
cls_id = classes.index(cls) # 排在第几位
xmlbox = obj.find('bndbox')
b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text),
float(xmlbox.find('ymax').text))
# 传入的是w,h 与框框的周边
bb = convert((w, h), b)
out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')


wd = getcwd()

for year, image_set in sets:
# ('2012', 'train') 循环5次
# 创建目录 一次性
if not os.path.exists('VOCdevkit/VOC%s/labels/' % (year)):
os.makedirs('VOCdevkit/VOC%s/labels/' % (year))

# 图片的id数据
image_ids = open('VOCdevkit/VOC%s/ImageSets/Main/%s.txt' % (year, image_set)).read().strip().split()

# 结果写入这个文件
list_file = open('%s_%s.txt' % (year, image_set), 'w')

for image_id in image_ids:
list_file.write('%s/VOCdevkit/VOC%s/JPEGImages/%s.jpg\n' % (wd, year, image_id)) # 补全路径
convert_annotation(year, image_id)
list_file.close()

  查看全部
作用读取每个xml文件,把坐标转化为相对坐标,对应文件名保存起来
 
import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import join




sets = [('2012', 'train'), ('2012', 'val'), ('2007', 'train'), ('2007', 'val'), ('2007', 'test')]

# 20类
classes = ["aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog",
"horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"]

# size w,h
# box x-min,x-max,y-min,y-max
def convert(size, box):
dw = 1. / size[0]
dh = 1. / size[1]
x = (box[0] + box[1]) / 2.0 # 中心点位置
y = (box[2] + box[3]) / 2.0
w = box[1] - box[0]
h = box[3] - box[2]
x = x * dw
w = w * dw
y = y * dh
h = h * dh # 全部转化为相对坐标
return (x, y, w, h)


def convert_annotation(year, image_id):
# 找到2个同样的文件
in_file = open('VOCdevkit/VOC%s/Annotations/%s.xml' % (year, image_id))
out_file = open('VOCdevkit/VOC%s/labels/%s.txt' % (year, image_id), 'w')

tree = ET.parse(in_file)
root = tree.getroot()
size = root.find('size')
w = int(size.find('width').text)
h = int(size.find('height').text)

for obj in root.iter('object'):
difficult = obj.find('difficult').text
cls = obj.find('name').text
if cls not in classes or int(difficult) == 1: # difficult ==1 的不要了
continue
cls_id = classes.index(cls) # 排在第几位
xmlbox = obj.find('bndbox')
b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text), float(xmlbox.find('ymin').text),
float(xmlbox.find('ymax').text))
# 传入的是w,h 与框框的周边
bb = convert((w, h), b)
out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')


wd = getcwd()

for year, image_set in sets:
# ('2012', 'train') 循环5次
# 创建目录 一次性
if not os.path.exists('VOCdevkit/VOC%s/labels/' % (year)):
os.makedirs('VOCdevkit/VOC%s/labels/' % (year))

# 图片的id数据
image_ids = open('VOCdevkit/VOC%s/ImageSets/Main/%s.txt' % (year, image_set)).read().strip().split()

# 结果写入这个文件
list_file = open('%s_%s.txt' % (year, image_set), 'w')

for image_id in image_ids:
list_file.write('%s/VOCdevkit/VOC%s/JPEGImages/%s.jpg\n' % (wd, year, image_id)) # 补全路径
convert_annotation(year, image_id)
list_file.close()

 

docker run 和 create 区别

Linux李魔佛 发表了文章 • 0 个评论 • 227 次浏览 • 2019-11-25 13:49 • 来自相关话题

Technically, docker run = docker create + docker start . 
docker create command creates a writeable container from the image and prepares it for running. 
docker run command creates the container (same as docker create ) and starts it.
 
  查看全部
Technically, docker run = docker create + docker start
docker create command creates a writeable container from the image and prepares it for running. 
docker run command creates the container (same as docker create ) and starts it.
 
 

用docker编译go代码

Linux李魔佛 发表了文章 • 0 个评论 • 174 次浏览 • 2019-11-25 13:45 • 来自相关话题

如果偶尔需要编译go代码,但是又不想要安装一堆乱七八糟的依赖和Go编译器,可以利用docker来实现。 应该是解决起来话费时间最小的。
1. 用文本编辑你的go代码,现在以hello world为例:
 package main import "fmt" func main() {
/* 这是我的第一个简单的程序 */
fmt.Println("Hello, World!")
}
 
2. 然后直接使用docker执行编译。docker首先会自动去下载go的编译器,顺便把所有的依赖给解决掉

docker run --rm -v "$(pwd)":/usr/src/hello -w /usr/src/hello golang:1.3 go build -v
 
最后会在本地生成一个编译好的hello静态文件。
上述docker命令的具体含义就是把当前路径挂在到docker容器里头,然后切换到改到改路径下,然后进行编译。 查看全部

如果偶尔需要编译go代码,但是又不想要安装一堆乱七八糟的依赖和Go编译器,可以利用docker来实现。 应该是解决起来话费时间最小的。
1. 用文本编辑你的go代码,现在以hello world为例:
 
package main import "fmt" func main() {
/* 这是我的第一个简单的程序 */
fmt.Println("Hello, World!")
}

 
2. 然后直接使用docker执行编译。docker首先会自动去下载go的编译器,顺便把所有的依赖给解决掉

docker run --rm -v "$(pwd)":/usr/src/hello -w /usr/src/hello golang:1.3 go build -v
 
最后会在本地生成一个编译好的hello静态文件。
上述docker命令的具体含义就是把当前路径挂在到docker容器里头,然后切换到改到改路径下,然后进行编译。