MyCaffe  1.12.2.41
Deep learning software for Windows C# programmers.
Public Member Functions | List of all members
MyCaffe.fillers.MsraFiller< T > Class Template Reference

Fills a Blob with values $ x \sim N(0, \sigma^2) $ where $ \sigma^2 $ is set inversely proportionla to number of incomming nodes, outgoing nodes, or their average. More...

Inheritance diagram for MyCaffe.fillers.MsraFiller< T >:
MyCaffe.fillers.Filler< T >

Public Member Functions

 MsraFiller (CudaDnn< T > cuda, Log log, FillerParameter p)
 Constructor. More...
 
override void Fill (int nCount, long hMem, int nNumAxes=1, int nNumOutputs=1, int nNumChannels=1, int nHeight=1, int nWidth=1)
 Fill the memory with random numbers from a MSRA distribution. More...
 
- Public Member Functions inherited from MyCaffe.fillers.Filler< T >
 Filler (CudaDnn< T > cuda, Log log, FillerParameter p)
 Constructor. More...
 
void Fill (Blob< T > b)
 Fill the blob with values based on the actual filler used. More...
 

Additional Inherited Members

- Static Public Member Functions inherited from MyCaffe.fillers.Filler< T >
static Filler< T > Create (CudaDnn< T > cuda, Log log, FillerParameter p)
 Create a new Filler instance. More...
 
- Protected Attributes inherited from MyCaffe.fillers.Filler< T >
CudaDnn< T > m_cuda
 Specifies the CudaDnn instance used to communicate to the low-level Cuda Dnn DLL. More...
 
Log m_log
 Specifies the output log. More...
 
FillerParameter m_param
 Specifies the filler parameters. More...
 

Detailed Description

Fills a Blob with values $ x \sim N(0, \sigma^2) $ where $ \sigma^2 $ is set inversely proportionla to number of incomming nodes, outgoing nodes, or their average.

A filler based on the paper Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification by He, Zhang, Ren and Sun 2015, which specifically accounts for ReLU nonlinearities.

Aside: for another perspective on the scaling factor, see the derivation of Learning hierarchical categories in deep neural networks by Saxe, McClelland, and Ganguli 2013 (v3).

It fills the incoming matrix by randomly sampling Gaussian data with $ std = sqrt(2 / n) $ where $ n $ is the fan_in, fan_out or their average, depending on the variance_norm option. You should make sure the input blob has shape (num, a, b, c) where $ a * b * c = fan_in $ and $ num * b * c = fan_out$. Note that this is currently not the case for inner product layers.

Template Parameters
TThe base type float or double.

Definition at line 30 of file MsraFiller.cs.

Constructor & Destructor Documentation

◆ MsraFiller()

MyCaffe.fillers.MsraFiller< T >.MsraFiller ( CudaDnn< T >  cuda,
Log  log,
FillerParameter  p 
)

Constructor.

Parameters
cudaInstance of CudaDnn - connection to cuda.
logLog used for output.
pFiller parameter that defines the filler settings.

Definition at line 38 of file MsraFiller.cs.

Member Function Documentation

◆ Fill()

override void MyCaffe.fillers.MsraFiller< T >.Fill ( int  nCount,
long  hMem,
int  nNumAxes = 1,
int  nNumOutputs = 1,
int  nNumChannels = 1,
int  nHeight = 1,
int  nWidth = 1 
)
virtual

Fill the memory with random numbers from a MSRA distribution.

Parameters
nCountSpecifies the number of items to fill.
hMemSpecifies the handle to GPU memory to fill.
nNumAxesOptionally, specifies the number of axes (default = 1).
nNumOutputsOptionally, specifies the number of outputs (default = 1).
nNumChannelsOptionally, specifies the number of channels (default = 1).
nHeightOptionally, specifies the height (default = 1).
nWidthOptionally, specifies the width (default = 1).

Implements MyCaffe.fillers.Filler< T >.

Definition at line 53 of file MsraFiller.cs.

The documentation for this class was generated from the following file: