MyCaffe
1.12.2.41
Deep learning software for Windows C# programmers.
|
The DataLayer loads data from the IXImageDatabase database. This layer is initialized with the MyCaffe.param.DataParameter. More...
Public Member Functions | |
DataLayer (CudaDnn< T > cuda, Log log, LayerParameter p, IXDatabaseBase db, CancelEvent evtCancel) | |
The DataLayer constructor. More... | |
void | Connect (DataLayer< T > src) |
The Connect method connects one Data Layer to another so that they can synchronize. More... | |
void | Disconnect () |
Disconnect any previously connected Data Layers. More... | |
Public Member Functions inherited from MyCaffe.layers.BasePrefetchingDataLayer< T > | |
BasePrefetchingDataLayer (CudaDnn< T > cuda, Log log, LayerParameter p, IXDatabaseBase db, CancelEvent evtCancel) | |
The BaseDataLayer constructor. More... | |
override void | LayerSetUp (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
LayerSetUp implements common data layer setup functonality, and calls DatLayerSetUp to do special data layer setup for individual layer types. More... | |
Public Member Functions inherited from MyCaffe.layers.BaseDataLayer< T > | |
BaseDataLayer (CudaDnn< T > cuda, Log log, LayerParameter p, IXDatabaseBase db) | |
The BaseDataLayer constructor. More... | |
override void | Reshape (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Data layers have no bottoms, so reshaping is trivial. More... | |
Public Member Functions inherited from MyCaffe.layers.Layer< T > | |
Layer (CudaDnn< T > cuda, Log log, LayerParameter p) | |
The Layer constructor. More... | |
void | Dispose () |
Releases all GPU and host resources used by the Layer. More... | |
virtual void | ConnectLoss (LossLayer< T > layer) |
Called to connect the loss OnLoss event to a specified layer (typically the data layer). More... | |
virtual BlobCollection< T > | PreProcessInput (PropertySet customInput, out int nSeqLen, BlobCollection< T > colBottom=null) |
The PreprocessInput allows derivative data layers to convert a property set of input data into the bottom blob collection used as intput. More... | |
virtual bool | PreProcessInput (string strEncInput, int? nDecInput, BlobCollection< T > colBottom) |
Preprocess the input data for the RUN phase. More... | |
virtual List< Tuple< string, int, double > > | PostProcessOutput (Blob< T > blobSofmtax, int nK=1) |
The PostProcessOutput allows derivative data layers to post-process the results, converting them back into text results (e.g., detokenizing). More... | |
virtual List< Tuple< string, int, double > > | PostProcessLogitsOutput (int nCurIdx, Blob< T > blobLogits, Layer< T > softmax, int nAxis, int nK=1) |
The PostProcessLogitsOutput allows derivative data layers to post-process the results, converting them back into text results (e.g., detokenizing). More... | |
virtual string | PostProcessFullOutput (Blob< T > blobSoftmax) |
The PostProcessFullOutput allows derivative data layers to post-process the results, usually be detokenizing the data in the blobSoftmax. More... | |
virtual string | PostProcessOutput (int nIdx) |
Convert the index to the word. More... | |
virtual void | SetOnDebug (EventHandler< GetWorkBlobArgs< T > > fn) |
Set the OnDebug event. More... | |
virtual void | ResetOnDebug (EventHandler< GetWorkBlobArgs< T > > fn) |
Reset the OnDebug event, disabling it. More... | |
virtual bool | ReInitializeParameters (WEIGHT_TARGET target) |
Re-initialize the parameters of the layer. More... | |
void | SetNetReshapeRequest () |
Called by the Net when requesting a reshape. More... | |
void | SetPhase (Phase phase) |
Changes the layer's Phase to the one specified. More... | |
void | Setup (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Implements common Layer setup functionality. More... | |
virtual void | SetNetParameterUsed (NetParameter np) |
This function allows other layers to gather needed information from the NetParameters if any, and is called when initialzing the Net. More... | |
void | ConvertToBase (BlobCollection< T > col) |
ConvertToBase converts any blobs in a collection that are in half size to the base size. More... | |
double | Forward (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Given the bottom (input) Blobs, this function computes the top (output) Blobs and the loss. More... | |
void | Backward (BlobCollection< T > colTop, List< bool > rgbPropagateDown, BlobCollection< T > colBottom) |
Given the top Blob error gradients, compute the bottom Blob error gradients. More... | |
double | loss (int nTopIdx) |
Returns the scalar loss associated with the top Blob at a given index. More... | |
void | set_loss (int nTopIdx, double dfLoss) |
Sets the loss associated with a top Blob at a given index. More... | |
virtual bool | AllowForceBackward (int nBottomIdx) |
Return whether to allow More... | |
bool | param_propagate_down (int nParamIdx) |
Returns whether or not the Layer should compute gradients w.r.t. a parameter at a particular index given by a parameter index. More... | |
void | set_param_propagate_down (int nParamIdx, bool bPropagate) |
Sets whether or not the Layer should compute gradients w.r.t. a parameter at a particular index given by a parameter index. More... | |
void | SetEnablePassthrough (bool bEnable) |
Enables/disables the pass-through mode. More... | |
Protected Member Functions | |
override void | preStop () |
The preStop override is called just before stopping the internal thread managed by the base class. More... | |
override void | dispose () |
Releases all GPU and host resources used by the Layer. More... | |
override void | DataLayerSetUp (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Setup the DataLayer by starting up the pre-fetching. More... | |
void | Next () |
Retrieves the next item from the database and rolls the cursor over once the end of the dataset is reached. More... | |
bool | Skip () |
Skip to the next value - used when training in a multi-GPU scenario. More... | |
override void | final_process (Blob< T > blobTop) |
Provides a final processing step that takes place at the end of the base class forward = this is where we apply the mask if one exists and is enabled. More... | |
override void | load_batch (Batch< T > batch) |
Load a batch of data in the background (this is run on an internal thread within the BasePrefetchingDataLayer class). More... | |
Protected Member Functions inherited from MyCaffe.layers.BasePrefetchingDataLayer< T > | |
void | statupPrefetch () |
Starts the prefetch thread. More... | |
override void | forward (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
The forward override implements the functionality to load pre-fetch data and feed it into the top (output) Blobs. More... | |
Protected Member Functions inherited from MyCaffe.layers.BaseDataLayer< T > | |
virtual bool | setupSourceDescriptor () |
Allows any derivative classes to pre-initialize the m_src which is used in LayerSetup before the DataLayerSetup. More... | |
override void | backward (BlobCollection< T > colTop, List< bool > rgbPropagateDown, BlobCollection< T > colBottom) |
Not implemented - data Layers do not perform backward. More... | |
Protected Member Functions inherited from MyCaffe.layers.Layer< T > | |
void | dispose (ref Layer< T > l) |
Helper method used to dispose internal layers. More... | |
void | dispose (ref Blob< T > b) |
Helper method used to dispose internal blobs. More... | |
void | dispose (ref BlobCollection< T > rg, bool bSetToNull=true) |
Dispose the blob collection. More... | |
GetIterationArgs | getCurrentIteration () |
Fires the OnGetIteration event to query the current iteration. More... | |
long | convert_to_full (int nCount, long hMem) |
Convert half memory to full memory. More... | |
void | convert (BlobCollection< T > col) |
Convert a collection of blobs from / to half size. More... | |
virtual bool | reshapeNeeded (BlobCollection< T > colBottom, BlobCollection< T > colTop, bool bReset=true) |
Tests the shapes of both the bottom and top blobs and if they are the same as the previous sizing, returns false indicating that no reshape is needed. More... | |
bool | compareShapes (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Compare the shapes of the top and bottom and if the same, return true, otherwise false. More... | |
void | setShapes (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Set the internal shape sizes - used when determining if a Reshape is necessary. More... | |
virtual void | setup_internal_blobs (BlobCollection< T > col) |
Derivative layers should add all internal blobws to the 'col' provided. More... | |
void | CheckBlobCounts (BlobCollection< T > colBottom, BlobCollection< T > colTop) |
Called by the Layer::Setup function to check the number of bottom (input) and top (output) Blobs provided match the expected number of blobs expected via the {EactNum,Min,Max}{Bottom,Top}Blobs functions. More... | |
void | SetLossWeights (BlobCollection< T > colTop) |
Called by Layer::Setup to initialize the weights associated with any top (output) Blobs in the loss function ans store non-zero loss weights in the diff Blob. More... | |
LayerParameter | convertLayerParam (LayerParameter pChild, LayerParameter pParent) |
Called to convert a parent LayerParameterEx, used in blob sharing, with a child layer parameter. More... | |
bool | shareParameter (Blob< T > b, List< int > rgMinShape, bool bAllowEndsWithComparison=false) |
Attempts to share a parameter Blob if another parameter Blob with the same name and accpetable size is found. More... | |
bool | shareLayerBlob (Blob< T > b, List< int > rgMinShape) |
Attempts to share a Layer Blob if another parameter Blob with the same name and acceptable size is found. More... | |
bool | shareLayerBlobs (Layer< T > layer) |
Attempts to share the Layer blobs and internal_blobs with matching names and sizes with those in another matching layer. More... | |
virtual WorkspaceArgs | getWorkspace () |
Returns the WorkspaceArgs used to share a workspace between Layers. More... | |
virtual bool | setWorkspace (ulong lSizeInBytes) |
Sets the workspace size (in items) and returns true if set, false otherwise. More... | |
void | check_nan (Blob< T > b) |
Checks a Blob for NaNs and throws an exception if found. More... | |
T | convert (double df) |
Converts a double to a generic. More... | |
T | convert (float f) |
Converts a float to a generic. More... | |
double | convertD (T df) |
Converts a generic to a double value. More... | |
float | convertF (T df) |
Converts a generic to a float value. More... | |
double[] | convertD (T[] rg) |
Converts an array of generic values into an array of double values. More... | |
T[] | convert (double[] rg) |
Converts an array of double values into an array of generic values. More... | |
float[] | convertF (T[] rg) |
Converts an array of float values into an array of generic values. More... | |
T[] | convert (float[] rg) |
Converts an array of float values into an array of generic values. More... | |
int | val_at (T[] rg, int nIdx) |
Returns the integer value at a given index in a generic array. More... | |
Size | size_at (Blob< T > b) |
Returns the Size of a given two element Blob, such as one that stores Blob size information. More... | |
Protected Attributes | |
DB< T > | m_db |
Specifies the database. More... | |
Cursor< T > | m_cursor |
Specifies the database used to traverse through the database. More... | |
Stopwatch | m_swTimerBatch |
Specifies a first timer used to calcualte the batch time. More... | |
Stopwatch | m_swTimerTransaction |
Specfies a second timer used to calculate the transaction time. More... | |
double | m_dfReadTime |
Specifies the read time. More... | |
double | m_dfTransTime |
Specifies the transaction time. More... | |
Protected Attributes inherited from MyCaffe.layers.BasePrefetchingDataLayer< T > | |
Batch< T >[] | m_rgPrefetch |
Specifies the pre-fetch cache. More... | |
CancelEvent | m_evtCancel |
Specifies the cancellation event for the internal thread. More... | |
Protected Attributes inherited from MyCaffe.layers.BaseDataLayer< T > | |
DataTransformer< T > | m_transformer |
Specifies the DataTransformer used to transform each data item as it loaded. More... | |
bool | m_bOutputLabels |
Specifies whether or not the Layer should output labels. More... | |
IXImageDatabaseBase | m_imgdb |
Specifies the MyCaffeImageDatabase. More... | |
SourceDescriptor | m_src |
Specifies the SourceDescriptor of the data source. More... | |
SimpleDatum | m_imgMean = null |
Specifies the SimpleDatum that optionally contains the image Mean for data centering. More... | |
Protected Attributes inherited from MyCaffe.layers.Layer< T > | |
LayerParameter.LayerType | m_type = LayerParameter.LayerType._MAX |
Specifies the Layer type. More... | |
CudaDnn< T > | m_cuda |
Specifies the CudaDnn connection to Cuda. More... | |
Log | m_log |
Specifies the Log for output. More... | |
LayerParameter | m_param |
Specifies the LayerParameter describing the Layer. More... | |
Phase | m_phase |
Specifies the Phase under which the Layer is run. More... | |
BlobCollection< T > | m_colBlobs |
Specifies the learnable parameter Blobs of the Layer. More... | |
BlobCollection< T > | m_colInternalBlobs = new BlobCollection<T>() |
Specifies internal blobs used by the layer. More... | |
DictionaryMap< bool > | m_rgbParamPropagateDown |
Specifies whether or not to compute the learnable diff of each parameter Blob. More... | |
DictionaryMap< double > | m_rgLoss |
Specifies the loss values that indeicate whether each top (output) Blob has a non-zero weight in the objective function.. More... | |
T | m_tOne |
Specifies a generic type equal to 1.0. More... | |
T | m_tZero |
Specifies a generic type equal to 0.0. More... | |
bool | m_bEnablePassthrough = false |
Enables/disables the pass-through mode for the layer. Default = false. More... | |
bool | m_bUseHalfSize = false |
Specifies that the half size of the top (if any) should be converted to the base size. More... | |
bool | m_bConvertTopOnFwd = false |
Specifies whether or not the layer should convert the top on the forward pass when using half sized memory (typically only done with input data). More... | |
bool | m_bConvertTopOnBwd = true |
Specifies whether or not to convert the top on the backward pass when using half sized memory (typically not done on loss layers). More... | |
bool | m_bConvertBottom = true |
Specifies whether or not the layer should convert the bottom when using half sized memory. More... | |
bool | m_bReshapeOnForwardNeeded = true |
Specifies whether or not the reshape on forward is needed or not. More... | |
bool | m_bNetReshapeRequest = false |
Specifies whether the reshape is requested from a Net.Reshape call or not. More... | |
LayerParameter.? LayerType | m_parentLayerType = null |
Specifies the layer type of the parent. More... | |
Properties | |
override bool | delayPrefetch [get] |
Specifies to delay the prefetch when using a synchronized Data Layer. More... | |
override int | ExactNumBottomBlobs [get] |
No bottom blobs are used by this layer. More... | |
override int | MinTopBlobs [get] |
Specifies the minimum number of required top (output) Blobs: data More... | |
override int | MaxTopBlobs [get] |
Specifies the maximum number of required top (output) Blobs: data, label More... | |
Properties inherited from MyCaffe.layers.BasePrefetchingDataLayer< T > | |
virtual bool | delayPrefetch [get] |
Specifies whether or not to delay the prefetch. More... | |
Properties inherited from MyCaffe.layers.BaseDataLayer< T > | |
SimpleDatum | ImageMean [getset] |
Get/set the image mean. More... | |
DataTransformer< T > | Transformer [get] |
Returns the data transformer used. More... | |
Properties inherited from MyCaffe.layers.Layer< T > | |
LayerParameter.? LayerType | parent_layer_type [get] |
Optionally, specifies the parent layer type (e.g. LOSS, etc.) More... | |
virtual bool | SupportsPreProcessing [get] |
Should return true when PreProcessing methods are overriden. More... | |
virtual bool | SupportsPostProcessing [get] |
Should return true when pre PostProcessing methods are overriden. More... | |
virtual bool | SupportsPostProcessingLogits [get] |
Should return true when pre PostProcessingLogits methods are overriden. More... | |
virtual bool | SupportsPostProcessingFullOutput [get] |
Should return true when PostProcessingFullOutput is supported. More... | |
BlobCollection< T > | blobs [get] |
Returns the collection of learnable parameter Blobs for the Layer. More... | |
BlobCollection< T > | internal_blobs [get] |
Returns the collection of internal Blobs used by the Layer. More... | |
LayerParameter | layer_param [get] |
Returns the LayerParameter for this Layer. More... | |
LayerParameter.LayerType | type [get] |
Returns the LayerType of this Layer. More... | |
virtual int | ExactNumBottomBlobs [get] |
Returns the exact number of bottom (input) Blobs required by the Layer, or -1 if no exact number is required. More... | |
virtual int | MinBottomBlobs [get] |
Returns the minimum number of bottom (input) Blobs required by the Layer, or -1 if no minimum number is required. More... | |
virtual int | MaxBottomBlobs [get] |
Returns the maximum number of bottom (input) Blobs required by the Layer, or -1 if no maximum number is required. More... | |
virtual int | ExactNumTopBlobs [get] |
Returns the exact number of top (output) Blobs required by the Layer, or -1 if no exact number is required. More... | |
virtual int | MinTopBlobs [get] |
Returns the minimum number of top (output) Blobs required by the Layer, or -1 if no minimum number is required. More... | |
virtual int | MaxTopBlobs [get] |
Returns the maximum number of top (output) Blobs required by the Layer, or -1 if no maximum number is required. More... | |
virtual bool | EqualNumBottomTopBlobs [get] |
Returns true if the Layer requires and equal number of bottom (input) and top (output) Blobs. More... | |
virtual bool | AutoTopBlobs [get] |
Return whether "anonymous" top (output) Blobs are created automatically by the Layer. More... | |
double | forward_timing [get] |
Returns the timing of the last forward pass in milliseconds. More... | |
double | forward_timing_average [get] |
Returns the average timing of the forward passes in milliseconds. More... | |
double | backward_timing [get] |
Returns the timing of the last backward pass in milliseconds. More... | |
double | backward_timing_average [get] |
Returns the average timing of the backward passes in milliseconds. More... | |
Events | |
EventHandler< LastBatchLoadedArgs > | OnBatchLoad |
This event fires (only when set) each time a batch is loaded form this dataset. More... | |
Events inherited from MyCaffe.layers.Layer< T > | |
EventHandler< WorkspaceArgs > | OnGetWorkspace |
Specifies the OnGetWorkspace event that fires when the getWorkspace() function is called by a layer to get a shareable workspace to conserve GPU memory. More... | |
EventHandler< WorkspaceArgs > | OnSetWorkspace |
Specifies the OnSetWorkspace event that fires when the setWorkspace() function is called by a layer to get a shareable workspace to conserve GPU memory. More... | |
EventHandler< GetIterationArgs > | OnGetIteration |
Specifies the OnGetIteration event that fires when a layer needs to get the current iteration from the solver. More... | |
EventHandler< GetWorkBlobArgs< T > > | OnDebug |
Specifies the OnGetWorkBlob event that is only supported when debugging to get a work blob from the primary Net holding this layer. More... | |
Additional Inherited Members | |
Static Public Member Functions inherited from MyCaffe.layers.Layer< T > | |
static Layer< T > | Create (CudaDnn< T > cuda, Log log, LayerParameter p, CancelEvent evtCancel, IXDatabaseBase db=null, TransferInput trxinput=null) |
Create a new Layer based on the LayerParameter. More... | |
The DataLayer loads data from the IXImageDatabase database. This layer is initialized with the MyCaffe.param.DataParameter.
T | Specifies the base type float or double. Using float is recommended to conserve GPU memory. |
Definition at line 23 of file DataLayer.cs.
MyCaffe.layers.DataLayer< T >.DataLayer | ( | CudaDnn< T > | cuda, |
Log | log, | ||
LayerParameter | p, | ||
IXDatabaseBase | db, | ||
CancelEvent | evtCancel | ||
) |
The DataLayer constructor.
cuda | Specifies the CudaDnn connection to Cuda. |
log | Specifies the Log for output. |
p | Specifies the LayerParameter data_param |
db | Specifies the external database to use. |
evtCancel | Specifies the CancelEvent used to cancel any pre-fetching operations. |
Definition at line 77 of file DataLayer.cs.
void MyCaffe.layers.DataLayer< T >.Connect | ( | DataLayer< T > | src | ) |
The Connect method connects one Data Layer to another so that they can synchronize.
src | Specifies the source Data Layer whos OnBatchLoad event fires and is handled by this Data Layer. |
Definition at line 189 of file DataLayer.cs.
|
protectedvirtual |
Setup the DataLayer by starting up the pre-fetching.
colBottom | Not used. |
colTop | Specifies the collection of top (output) Blobs. |
Implements MyCaffe.layers.BaseDataLayer< T >.
Definition at line 220 of file DataLayer.cs.
void MyCaffe.layers.DataLayer< T >.Disconnect | ( | ) |
Disconnect any previously connected Data Layers.
Definition at line 201 of file DataLayer.cs.
|
protectedvirtual |
Releases all GPU and host resources used by the Layer.
Reimplemented from MyCaffe.layers.BasePrefetchingDataLayer< T >.
Definition at line 140 of file DataLayer.cs.
|
protectedvirtual |
Provides a final processing step that takes place at the end of the base class forward = this is where we apply the mask if one exists and is enabled.
blobTop | Specifies the top blob just about to be set out the forward operation as the Top[0] blob. |
Reimplemented from MyCaffe.layers.BasePrefetchingDataLayer< T >.
Definition at line 526 of file DataLayer.cs.
|
protectedvirtual |
Load a batch of data in the background (this is run on an internal thread within the BasePrefetchingDataLayer class).
batch | Specifies the Batch of data to load. |
Implements MyCaffe.layers.BasePrefetchingDataLayer< T >.
Definition at line 595 of file DataLayer.cs.
|
protected |
Retrieves the next item from the database and rolls the cursor over once the end of the dataset is reached.
Definition at line 495 of file DataLayer.cs.
|
protectedvirtual |
The preStop override is called just before stopping the internal thread managed by the base class.
Reimplemented from MyCaffe.layers.BasePrefetchingDataLayer< T >.
Definition at line 131 of file DataLayer.cs.
|
protected |
Skip to the next value - used when training in a multi-GPU scenario.
Definition at line 512 of file DataLayer.cs.
|
protected |
Specifies the database used to traverse through the database.
Definition at line 32 of file DataLayer.cs.
|
protected |
Specifies the database.
Definition at line 28 of file DataLayer.cs.
|
protected |
Specifies the read time.
Definition at line 45 of file DataLayer.cs.
|
protected |
Specifies the transaction time.
Definition at line 49 of file DataLayer.cs.
|
protected |
Specifies a first timer used to calcualte the batch time.
Definition at line 37 of file DataLayer.cs.
|
protected |
Specfies a second timer used to calculate the transaction time.
Definition at line 41 of file DataLayer.cs.
|
getprotected |
Specifies to delay the prefetch when using a synchronized Data Layer.
Definition at line 172 of file DataLayer.cs.
|
get |
No bottom blobs are used by this layer.
Definition at line 470 of file DataLayer.cs.
|
get |
Specifies the maximum number of required top (output) Blobs: data, label
Definition at line 486 of file DataLayer.cs.
|
get |
Specifies the minimum number of required top (output) Blobs: data
Definition at line 478 of file DataLayer.cs.
EventHandler<LastBatchLoadedArgs> MyCaffe.layers.DataLayer< T >.OnBatchLoad |
This event fires (only when set) each time a batch is loaded form this dataset.
Definition at line 67 of file DataLayer.cs.