I expect you’ve come across this situation too while working on your Machine Learning projects. You’re making progress each day and feeling proud of what you’ve done. Until you don’t.

Yes, this is when something unexpected happens and we hit another roadblock, another error. It happened to me in the weekend when I ran my Binary Classification Sequential Deep Learning model for the first time. I “cheated” and used a Mobilenet v2 algorithm that’s been trained on a huge dataset of images. This way of doing is called Transfer Learning and it saves time and computing power.

I used “Sigmoid” activation and “Binary cross-entropy” as my loss function. Luckily I have a great teacher who uses analogies to explain complex ideas. His way of teaching “Loss” is to imagine we’re at the top of a hill about to ski down it BLINDFOLDED in the International Downhill Skiing Competition. Adam is our coach at the bottom yelling out instructions so we find the quickest route to the finishing line. And “accuracy” is our judge scoring how well we did.

Anyway I digress. It didn’t take long for my model to hit a snag. The message read “trainjpg/202206113.jpg – there is no such file or directory”. Damn. It’s so tempting to throw a tantrum and cry “It’s not fair” or “Why is this so hard” or even “Just give me a break”. We have to remember that this is another opportunity for us to learn something we didn’t know before. To become a detective and figure out how to fix it.

Like a favorite character of mine, Precious Ramotswe from Alexander McCall Smith’s series of novels featuring The Number 1 Ladies Detective Agency.

I found the image that was missing by creating data frames from my two lists and then comparing them using a Numpy function np.setdiff1d().Then I deleted the corresponding image label in my label.csv file. I thought I’d fixed my problem. Surely this would mean that I have the same number of filenames as images now? No. Somehow the length of my “trainjpg” list was still too long.

To cut a long story short I found my logs directory for TensorBoard had ended up in the wrong folder directory and that’s where the two extra things were. Also I learned about hidden .ipynb_checkpoint folders and how to delete them. I also discovered I had put my logs directory in the wrong place and that was causing issues too. Finally I got my directories tidied up and checked to see if I had the same number of images, labels and filenames again. YES!!

Finally I could try running my model again. This was the exciting bit. I watched it run and get better with time. There were thirteen batches of 32 image-label tensor pairs and with my Epoch Length set at a maximum of 100 I wondered how many it would take before my early stopping callback kicked in.

Check out my next blog post to see how many it took and how well my adapted mobilenet_v2 model performed. With the right attitude we can view obstacles as educational opportunities. They build confidence as we figure them out and you can never have enough of that.