Accuracy Issue : Not achieved as displayed

Accuracy Issue : Not achieved as displayed

I have completed training using alexnet and googlenet:

Training Accuracy Alexnet : 33%

Training Accuracy GoogleNet : 100% . 

As i got 100%, i shocked and tested manually, but it is not giving me desired answer, as well as as per the dataset and result of alexnet accuracy should not go beyond 45-48%.

For all result (out of 5 classes), it is giving me mis-classification to class 4 and then 1. Snapshot of those attached herewith.

GoogleNet Results ACCURACY 100%

Misclassification of Class 3 to class 4, 1

Misclassification of Class 5 to class 4,1

 

 

 

 

13 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Hello,

I have kind of the same issue

1 .I uploaded the 101Objcets data and configured the dataset and proceeded to training models with each of the available algorithms.

example of the GoogLeNet topology

Topology:

 
Parameters:
 

GoogLeNet
Training epochs: 60
Snapshot interval (in epochs): 1

Dataset:
Validation Interval: 1

Name: 101_objects
Batch size: 32
Validation batch size: 50

Number of Categories: 102
Solver type: SGD
Learning Rate: 0.01

Time Submitted: 15:42:53
Policy: poly
Weight Deacay: 0.0002

Dataset size: 108.403 MB
Crop Size: (no information)
Resize transformation: fill

Backend: LEVELDB
Subtract mean: (no information)
Regularization type: L2

 
 
 

General Information:
Image:
Advanced options:

Model Name: Bel TSC google 2
Dimensions: 256 x 256
Step Size: 0 (no information)

Description:
Type: Color images
Gamma: 0 (no information)

 

Encoding: none
Power: 0.5

 
 
Solver type: SGD

 
 
momentum: 0.9

2. I observe that in the data set validation tab everything seems fine the objects are identified correctly.

3. However when I move to testing of the model the result are all wrong, 90% of the time the given object is not in the best out of 6 confidence list.

4. This also happens the other topologies.

5. While training the Accuracy converged to 100% and Training Loss to ~0 very fast (5% of the training time), this seems odd.

Regards,

Mihai

 

 

safvan v, mihai d : May I know for which framework you are observing this inaccuracy issue ? (caffe/ tensorflow?).

I think caffe only, i don't know how to choose tensor flow in intel deep learning sdk. Please guide us.

I ran the training again, and this time for the 101Objects everything is detected 100% as BACKGROUND_Google and everything else is 0% when testing. While in the Dataset->Validation tab I see multiple classes being recognized.

The default framework for the provided topologies is caffe right? Tensorflow is only available in the jupyter app, can anyone confirm this?

Right, the default framework is Caffe.

Tensorflow is not supported in the Beta release of the training tool (it's not installed in the docker container).

Any solution for the given problem.

One more, that should i have to resize while testing the image? or model itself will take care about this and resize image before testing.

how are you using the dataset of 101_objects?It seems while training I get an error always.I am using VM instance.I am unable to train the data from the objects.Did you resize it or kept it as default?

Hi,

Currently we have a bug with the testing capability (scoring a single image)

This bug is going to be fixed for the next version.

To this moment you can implement your custom testing capability using the jupyter notebook.

Regards,

Yevgeni.

Quote:

Abhishek 81 wrote:

how are you using the dataset of 101_objects?It seems while training I get an error always.I am using VM instance.I am unable to train the data from the objects.Did you resize it or kept it as default?

Please check what is the error about, you can do this by clicking the "Click to download log" link near the failed model.

Yevgeni.

I am going through the doc file but I am unable to focus which file is the important one and why the training model failed.I am sharing the file

Attachments: 

AttachmentSize
Downloadapplication/zip model(2).zip4.51 KB

Dear all,

I had the same issue. Then I figured out that the DLSDK interface is buggy and doesn't work correctly. The only way to test your model correctly is to use Jupyter+python to test your model. The SDK provide a sample python notebook under the advanced tab. You can use the file "00-classification.ipynb" and modify it to work with your own model and sample images. The modifications are not much. 

With this method you can correctly test your model and visualize any part of it as well.

If you need help to modify the python code I will be happy to help!

Edit: You have to note that the Jupyter notebook is running from the docker machine. So all paths are relative to the docker machine not the host computer. 

You can start a shell session for your docker machine using the following command:

> sudo docker exec -it trainingtool "bash"

 

Thanks for input & help :)

We do have a bug in the inference and it will be resolved in our next version (beginning of April).

You can also open a terminal form the "Advanced" section (the Jupyter notebooks) - under new - Terminal. It will open the terminal inside the docker.

Leave a Comment

Please sign in to add a comment. Not a member? Join today