We have previously discussed that we are conducting experiments using the MNIST dataset, and released the code for the MNIST and NIST preprocessing code. For the next phase of our experiments, we have begun experimenting with the Street View House Numbers (SVHN) dataset to test the robustness of our algorithms.
The SVHN dataset contains real world images obtained from the house numbers in Google Street View images. The dataset comes in a similar style as the MNIST dataset where images are of small cropped digits, while being significantly harder and containing an order of magnitude more labelled data.
The commonly used version of the dataset (Format 2) comes in the MAT file format which contains a 4D matrix (X) for the features and a vector (y) for the labels. There are several available packages, such as SciPy, that can easily load and use the data from MAT files. However, to maintain a consistent data structure and easily switch between datasets in our experiments, we needed to preprocess the data in a similar fashion to how we preprocessed the MNIST images.
If you want to consume the dataset as standard images this would be useful for you too.
We created a simple and self-contained Python project with a small number of dependencies that can be easily installed using pip. The script takes in the input path to the MAT file and output path to save the images. There is also an option to convert the images to grayscale. See the README for more information on the usage.
Just yesterday I started looking for the Mnist dataset, so you reminded me of your preprocessing code in the nick of time. Unfortunately I have no idea about java and haven’t managed to compile the code.
I did manage to compile the CLIParser and MnistDataFile classes, but the MnistPreprocess.java somehow doesn’t find them anyway:
… src/mnist/MnistPreprocess.java:267: error: package CLISimpleParser does not exist
for( CLISimpleParser.ArgumentEntry argumentEntry : argumentEntries ) {
^
21 errors
I did just manage to extract the tensorflow Mnist dataset, so I’m good. But maybe update the ReadMe anyway for other people similarly inept.