在使用gluon时 怎么gradient clipping?

如题目,谢谢

gluon.Trainer里面有这个参数
https://mxnet.incubator.apache.org/versions/master/api/python/gluon/gluon.html#mxnet.gluon.Trainer
比如

gluon.Trainer(net.collect_params(), 'sgd', {'lr': 1e-2, 'grad_clip': 2})
2赞

非常感谢

这个是global的还是每个梯度的最大值?

我这边出现了这样的错误
File “main_task1.py”, line 95, in custom_fit
optimizer_params=optimizer_params, kvstore=kvstore)
File “/scratch/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/mxnet-20171005.1-gjvqsive3w4trrb2mpsljgut7cacoqgg/lib/python2.7/site-packages/
mxnet/gluon/trainer.py”, line 72, in init
self._init_optimizer(optimizer, optimizer_params)
File “/scratch/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/mxnet-20171005.1-gjvqsive3w4trrb2mpsljgut7cacoqgg/lib/python2.7/site-packages/
mxnet/gluon/trainer.py”, line 97, in _init_optimizer
**optimizer_params)
File “/scratch/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/mxnet-20171005.1-gjvqsive3w4trrb2mpsljgut7cacoqgg/lib/python2.7/site-packages/
mxnet/optimizer.py”, line 160, in create_optimizer
return Optimizer.opt_registryname.lower()
File “/scratch/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/mxnet-20171005.1-gjvqsive3w4trrb2mpsljgut7cacoqgg/lib/python2.7/site-packages/
mxnet/optimizer.py”, line 669, in init
super(AdaGrad, self).init(**kwargs)
TypeError: init() got an unexpected keyword argument ‘grad_clip’

代码是
trainer = gluon.Trainer(self.collect_params(), optimizer=optimizer,
optimizer_params={‘learning_rate’: args.lr,
‘momentum’: args.mom,
‘wd’: args.wd,
# ‘clip_gradient’: args.clip,
‘grad_clip’: args.clip
}, kvstore=kvstore)

你改成clip_gradient可以吗

我不是很确定我的gradient 是不是被clip了……设成了了1e-16网络结果还是会动……

你可以把梯度打出来看看,比如说

loss.backward()
print(params.grad())

看看梯度具体是多少

mxnet==1.5.1,‘grad_clip’参数改为了‘clip_gradient’参数了