softmax回归的简洁实现 讨论区


#22

还是报同样的错,我的代码是

def transform(data, label):
    return data.astype('float32')/255, label.astype('float32')
mnist_train = gluon.data.vision.FashionMNIST(train=True, transform=transform)
mnist_test = gluon.data.vision.FashionMNIST(train=False, transform=transform)

batch_size = 256
train_data = gluon.data.DataLoader(mnist_train, batch_size, shuffle=True)
test_data = gluon.data.DataLoader(mnist_test, batch_size, shuffle=False)

net = gluon.nn.Sequential()
with net.name_scope():
    net.add(gluon.nn.Flatten())
    net.add(gluon.nn.Dense(10))
net.initialize()

net.collect_params().reset_ctx(ctx=mx.gpu())
softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss()
trainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': 0.1})

for epoch in range(10):
    tr_loss = 0
    tr_acc = 0
    for data, label in train_data:
        with autograd.record():
            output = net(data)
            loss = softmax_cross_entropy(output, label)
        loss.backward()
#         average lr, so it is insensitive to batch_size?
        trainer.step(batch_size)
        
        tr_loss += nd.mean(loss).asscalar()
        tr_acc += accuracy(output, label)
        
    test_acc = evaluate_accuracy(test_data, net)
    print("epoch %d, loss %f, train acc %f, test acc %f" % (
        epoch, tr_loss/len(train_data), tr_acc/len(train_data), test_acc))

#23

把错误也贴一下吧


#24

看起来像是把某个路径导入到解析器环境中,便于import 模块


#25
---------------------------------------------------------------------------
DeferredInitializationError               Traceback (most recent call last)
/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/block.py in forward(self, x, *args)
    422                 try:
--> 423                     params = {i: j.data(ctx) for i, j in self._reg_params.items()}
    424                 except DeferredInitializationError:

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/block.py in <dictcomp>(.0)
    422                 try:
--> 423                     params = {i: j.data(ctx) for i, j in self._reg_params.items()}
    424                 except DeferredInitializationError:

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/parameter.py in data(self, ctx)
    338         """
--> 339         return self._check_and_get(self._data, ctx)
    340 

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/parameter.py in _check_and_get(self, arr_dict, ctx)
    148         if self._deferred_init:
--> 149             raise DeferredInitializationError
    150         raise RuntimeError(

DeferredInitializationError: 

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
<ipython-input-28-898acf014b76> in <module>()
      4     for data, label in train_data:
      5         with autograd.record():
----> 6             output = net(data)
      7             loss = softmax_cross_entropy(output, label)
      8         loss.backward()

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/block.py in __call__(self, *args)
    285     def __call__(self, *args):
    286         """Calls forward. Only accepts positional arguments."""
--> 287         return self.forward(*args)
    288 
    289     def forward(self, *args):

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/nn/basic_layers.py in forward(self, x)
     44     def forward(self, x):
     45         for block in self._children:
---> 46             x = block(x)
     47         return x
     48 

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/block.py in __call__(self, *args)
    285     def __call__(self, *args):
    286         """Calls forward. Only accepts positional arguments."""
--> 287         return self.forward(*args)
    288 
    289     def forward(self, *args):

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/block.py in forward(self, x, *args)
    426                     for i in self.collect_params().values():
    427                         i._finish_deferred_init()
--> 428                     params = {i: j.data(ctx) for i, j in self._reg_params.items()}
    429                 return self.hybrid_forward(ndarray, x, *args, **params)
    430 

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/block.py in <dictcomp>(.0)
    426                     for i in self.collect_params().values():
    427                         i._finish_deferred_init()
--> 428                     params = {i: j.data(ctx) for i, j in self._reg_params.items()}
    429                 return self.hybrid_forward(ndarray, x, *args, **params)
    430 

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/parameter.py in data(self, ctx)
    337         NDArray on ctx
    338         """
--> 339         return self._check_and_get(self._data, ctx)
    340 
    341     def list_data(self):

/home/yzhao30/anaconda3/lib/python3.6/site-packages/mxnet/gluon/parameter.py in _check_and_get(self, arr_dict, ctx)
    145                 "Parameter %s was not initialized on context %s. "
    146                 "It was only initialized on %s."%(
--> 147                     self.name, str(ctx), str(self._ctx_list)))
    148         if self._deferred_init:
    149             raise DeferredInitializationError

RuntimeError: Parameter sequential3_dense0_weight was not initialized on context cpu(0). It was only initialized on [gpu(0)].

#26

把这两句

net.initialize()
net.collect_params().reset_ctx(ctx=mx.gpu())

改成

net.initialize(ctx=mx.gpu())

应该就可以了


#27
  1. net.initialize(mx.gpu())
  2. 训练时从train_data当中获得data和label后, 加上 data = data.as_in_context(mx.gpu()); label = label.as_in_context(mx.gpu()); 之后再将data和label用于net和计算loss

#29

多谢,问题解决了。


#30

AttributeError Traceback (most recent call last)
in ()
64
65 train_loss += nd.mean(loss).asscalar()
—> 66 train_acc += accuracy(output, label)
67
68 test_acc = evaluate_accuracy(test_data, net)

in accuracy(output, label)
8
9 def accuracy(output, label):
—> 10 return nd.mean(output.argmax(axis=1)==label).asscalar()
11
12 def evaluate_accuracy(data_iterator, net, ctx=mx.cpu()):

AttributeError: ‘NDArray’ object has no attribute ‘argmax’


#31

想问下课程中的问题。为什么参数都差不多,但gluon版本比从0开始的版本精度更高?


#32

我的理解是有2个原因:
1.net.initialize():自带的初始化可能跟我们的随机初始化不完全一样,可能是更合适的(希望有人可以解答)。
2.softmax_cross_entropy = gluon.loss.SoftmaxCrossEntropyLoss() 将softmax与交叉熵放在一起,数值更加稳定,也就降低了之前对softmax函数直接求导可能带来的风险。
不知道我的理解对不对,如果有错误,希望指出。


#33

请问

with net.name_scope():   
    net.add(gluon.nn.Flatten())
    net.add(gluon.nn.Dense(10))

net.add(gluon.nn.Flatten())
net.add(gluon.nn.Dense(10))

的区别在哪呢?(或者说net.name_scope()的功能是什么)
我看了下API中的意思:
Returns a name space object managing a child Block and parameter names. Should be used within a with statement
(但不太清楚,print(net)的结果也是一样的)
block.py中看是返回:self._scope = _BlockScope(self)
这部分的作用具体是什么呢?

谢谢~


#34

用name_scope()可以手动定义名字。在load/save的时候会方便点。但大部分时候用户不用管这个。。

http://zh.gluon.ai/chapter_gluon-basics/block.html#使用-nn.Block-来定义


#35


最近自己复写了教程里面的模型,发现个奇怪的问题,之前学习的时候运行并没有问题,现在报这个异常是什么原因?
打印出来idx.shape,X.shape[0]是image,迭代的第一次就报错了。没理解为啥报错。。。


#36

flatten扁平化是这么个意思,?


#37

对的,是这样意思


#38

调了一下Learning rate, 发现LR调大收敛的快,但是精度不能达到很高,LR调小,收敛的慢,但是能够达到比较高的精度,请问如何选取一个较优的LR,使得收敛的速度即快,达到的精度也高?


#39

这个问题我也遇到了,应该是因为教程在更新的时候utils下的load_data_fashion_mnist函数内部发生了变化,如果是以前的版本是可以迭代得到结果的。因为后面的内容我也还没看到,所以不知道后期对这个函数做了什么改进,但是做模型训练的话可以先通过从零开始的载入数据的方式对数据进行导入,然后就可以用gluon训练了


#40

增大学习率后(即使到1000)都不会出现手工写的会出现的数值不稳定问题,只是在相同较小的的epoch下,精度比0.1稍微低一点。

为什么参数差不多,gluon版本比手工写的精度要高?

数据处理一样
模型不一样 loss不同
优化方式都是SGD

所以是loss 导致的差别, 具体是?


#41

我想问下,这个为什么使用MXNet,现在不是tensorflow更好?还是因为tensorflow都集成好了,老师让我们了解更深入一些所以用MXNET的GLUON??


#42

Gluon编程更简单。新的TF Eager可能跟Gluon类似