Designing a Convolutional Neural Network for Image Recognition: A Comparative Study of Different Architectures and Training Techniques

28 Pages Posted: 27 Feb 2023

Tapomoy Adhikari

Microsoft Corporation

Date Written: February 22, 2023

A powerful tool for image recognition, Convolutional Neural Networks (CNNs) have been successfully applied in various fields including computer vision, medical image analysis, and self- driving cars. However, when dealing with large datasets, selecting the right architecture and training technique for a CNN can be challenging. This thesis aims to identify the most effective approach for image recognition by comparing different CNN architectures and training techniques. The literature review provides an overview of CNNs for image recognition, discussing various architectures and training techniques that have been used in previous studies. The review explains common CNN architectures such as LeNet, AlexNet, VGG, and ResNet, highlighting their strengths and weaknesses. The literature also covers popular training techniques, including SGD, Adam, and BN. The study used the CIFAR-10 dataset, comprising 60,000 color images of 32x32 pixels, classified into ten different classes. The data was preprocessed by normalizing pixel values and augmenting the training set with random flips and rotations. The researchers implemented and trained seven different CNN architectures, including LeNet, AlexNet, VGG-16, VGG-19, ResNet-50, ResNet-101, and ResNet-152, using three different training techniques: SGD, Adam, and SGD with BN. The results show that ResNet-152 was the most effective architecture for the CIFAR-10 dataset, achieving an accuracy of 94.7%. ResNet-101 and VGG-19 followed closely, both achieving an accuracy of 93.7%. Deeper networks performed better than shallower ones, with ResNet-152, which has 152 layers, outperforming VGG-19, which has 19 layers. Adding BN to the SGD training technique improved the performance of the CNN architectures, resulting in higher accuracy and faster convergence. This comparative study provides valuable insights into the performance of different CNN architectures and training techniques for image recognition. The findings demonstrate the importance of selecting the appropriate CNN architecture and training technique for achieving high accuracy in image recognition tasks. The study highlights the importance of the training technique, with the addition of BN to SGD resulting in improved performance. The implications of these findings are practical, as they could assist researchers and practitioners in the field of image recognition in designing CNNs. Future research could investigate the performance of these CNN architectures and training techniques on other datasets and explore other state-of-the-art techniques, such as transfer learning and adversarial training. This study's findings have potential for wider application in various fields where CNNs are used for image recognition, including self-driving cars, medical image analysis, and computer vision.

Keywords: Convolutional Neural Network, Machine Learning

Suggested Citation: Suggested Citation

Tapomoy Adhikari (Contact Author)

Microsoft corporation ( email ), do you have a job opening that you would like to promote on ssrn, paper statistics, related ejournals, applied computing ejournal.

Subscribe to this fee journal for more curated articles on this topic

Computation Theory eJournal

Electrical engineering ejournal, computer science educator: courses, cases & teaching ejournal, bioengineering ejournal, automotive engineering ejournal.

convolutional neural network thesis pdf

Master Thesis on Bayesian Convolutional Neural Network using Variational Inference

kumar-shridhar/Master-Thesis-BayesianCNN

Name already in use.

Use Git or checkout with SVN using the web URL.

Work fast with our official CLI. Learn more .

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Master thesis: bayesian convolutional neural networks.

Thesis work submitted at Computer Science department at University of Kaiserslautern.

License MIT

Supervisors

Artificial Neural Networks are connectionist systems that perform a given task by learning on examples without having a prior knowledge about the task. This is done by finding an optimal point estimate for the weights in every node. Generally, the network using point estimates as weights perform well with large datasets, but they fail to express uncertainty in regions with little or no data, leading to overconfident decisions.

In this thesis, Bayesian Convolutional Neural Network (BayesCNN) using Variational Inference is proposed, that introduces probability distribution over the weights. Furthermore, the proposed BayesCNN architecture is applied to tasks like Image Classification, Image Super-Resolution and Generative Adversarial Networks.

BayesCNN is based on Bayes by Backprop which derives a variational approximation to the true posterior. Our proposed method not only achieves performances equivalent to frequentist inference in identical architectures but also incorporate a measurement for uncertainties and regularisation. It further eliminates the use of dropout in the model. Moreover, we predict how certain the model prediction is based on the epistemic and aleatoric uncertainties and finally, we propose ways to prune the Bayesian architecture and to make it more computational and time effective.

In the first part of the thesis, the Bayesian Neural Network is explained and it is applied to an Image Classification task. The results are compared to point-estimates based architectures on MNIST, CIFAR-10, CIFAR-100 and STL-10 datasets. Moreover, uncertainties are calculated and the architecture is pruned and a comparison between the results is drawn.

In the second part of the thesis, the concept is further applied to other computer vision tasks namely, Image Super-Resolution and Generative Adversarial Networks. The concept of BayesCNN is tested and compared against other concepts in a similar domain.

The proposed work has been implemented in PyTorch and is available here : BayesianCNN

Chapter Overview

Chapter 1 : introduction.

Why there is a need for Bayesian Networks?

Problem Statement

Current Situation

Our Hypothesis

Our Contribution

Chapter 2: Background

Neural Networks and Convolutional Neural Networks

Concepts overview of Variational Inference, and local reparameterization trick in Bayesian Neural Network.

Backpropagation in Bayesian Networks using Bayes by Backprop.

Estimation of Uncertainties in a network.

Pruning a network to reduce the number of overall parameters without affecting it's performance.

Chapter 3: Related Work

How Bayesian Methods were applied to Neural Networks for the intractable true posterior distribution.

Various ways of training Neural Networks posterior probability distributions: Laplace approximations, Monte Carlo and Variational Inference.

Proposals on Dropout and Gaussian Dropout as Variational Inference schemes.

Work done in the past for uncertainty estimation in Neural Network.

Ways to reduce the number of parameters in a model.

Chapter 4: Concept

Bayesian CNN with Variational Inference based on Bayes by Backprop.

Bayesian convolutional operations with mean and variance.

Local reparameterization trick for Bayesian CNN.

Uncertainty estimation in a Bayesian network.

Using L1 norm for reducing the number of parameters in a Bayesian network.

Chapter 5: Empirical Analysis

Applying Bayesian CNN for the task of Image Recognition on MNIST, CIFAR-10, CIFAR-100 and STL-10 datasets.

Comparison of results of Bayesian CNN with Normal CNN architectures on similar datasets.

Regularization effect of Bayesian Network with dropouts.

Distribution of mean and variance in Bayesian CNN over time.

Parameters comparison before and after model pruning.

Chapter 6: Applications

Empirical analysis of BayesCNN with normal architecture for Image Super Resolution.

Empirical analysis of BayesCNN with normal architecture for Generative Adversarial Networks.

Chapter 7: Conclusion and Outlook

Journal paper of this work is also available on Arxiv: A Comprehensive guide to Bayesian Convolutional Neural Network with Variational Inference

Feel free to cite, if the work is of any help to you:

Thesis Template

Contributors 2

Information

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature Papers represent the most advanced research with significant potential for high impact in the field. Feature Papers are submitted upon individual invitation or recommendation by the scientific editors and undergo peer review prior to publication.

The Feature Paper can be either an original research article, a substantial novel research study that often involves several techniques or approaches, or a comprehensive review paper with concise and precise updates on the latest progress in the field that systematically reviews the most exciting advances in scientific literature. This type of paper provides an outlook on future directions of research or possible applications.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

convolutional neural network thesis pdf

applsci-logo

Article Menu

convolutional neural network thesis pdf

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Convolutional neural network-based soil water content and density prediction model for agricultural land using soil surface images, 1. introduction, 2. materials and methods, 2.1. soil surface image acquisition, 2.2. data preparation for the cnn model, 2.3. convolutional neural network, 3.1. quality of original and segmented images, 3.2. output of cnn model, 3.3. accuuracy of soil water content and soil dry density prediction, 4. discussion, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

Share and Cite

Kim, D.; Kim, T.; Jeon, J.; Son, Y. Convolutional Neural Network-Based Soil Water Content and Density Prediction Model for Agricultural Land Using Soil Surface Images. Appl. Sci. 2023 , 13 , 2936. https://doi.org/10.3390/app13052936

Kim D, Kim T, Jeon J, Son Y. Convolutional Neural Network-Based Soil Water Content and Density Prediction Model for Agricultural Land Using Soil Surface Images. Applied Sciences . 2023; 13(5):2936. https://doi.org/10.3390/app13052936

Kim, Donggeun, Taejin Kim, Jihun Jeon, and Younghwan Son. 2023. "Convolutional Neural Network-Based Soil Water Content and Density Prediction Model for Agricultural Land Using Soil Surface Images" Applied Sciences 13, no. 5: 2936. https://doi.org/10.3390/app13052936

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

MDPI

Subscribe to receive issue release notifications and newsletters from MDPI journals

YouBookinc.com

Tree based convolutional neural networks.

Download Tree Based Convolutional Neural Networks full books in PDF, epub, and Kindle. Read online free Tree Based Convolutional Neural Networks ebook anywhere anytime directly on your device. Fast Download speed and no annoying ads. We cannot guarantee that every ebooks is available!

Tree Based Convolutional Neural Networks

Download Tree Based Convolutional Neural Networks Book in PDF, Epub and Kindle

This book proposes a novel neural architecture, tree-based convolutional neural networks (TBCNNs),for processing tree-structured data. TBCNNsare related to existing convolutional neural networks (CNNs) and recursive neural networks (RNNs), but they combine the merits of both: thanks to their short propagation path, they are as efficient in learning as CNNs; yet they are also as structure-sensitive as RNNs. In this book, readers will also find a comprehensive literature review of related work, detailed descriptions of TBCNNs and their variants, and experiments applied to program analysis and natural language processing tasks. It is also an enjoyable read for all those with a general interest in deep learning.

Advances in Knowledge Discovery and Data Mining

Advances in Knowledge Discovery and Data Mining

Download Advances in Knowledge Discovery and Data Mining Book in PDF, Epub and Kindle

This two-volume set, LNAI 10234 and 10235, constitutes the thoroughly refereed proceedings of the 21st Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, PAKDD 2017, held in Jeju, South Korea, in May 2017. The 129 full papers were carefully reviewed and selected from 458 submissions. They are organized in topical sections named: classification and deep learning; social network and graph mining; privacy-preserving mining and security/risk applications; spatio-temporal and sequential data mining; clustering and anomaly detection; recommender system; feature selection; text and opinion mining; clustering and matrix factorization; dynamic, stream data mining; novel models and algorithms; behavioral data mining; graph clustering and community detection; dimensionality reduction.

Smart Computing and Communication

Smart Computing and Communication

Download Smart Computing and Communication Book in PDF, Epub and Kindle

This book constitutes the refereed proceedings of the 4th International Conference on Smart Computing and Communications, SmartCom 2019, held in Birmingham, UK, in October 2019. The 40 papers presented in this volume were carefully reviewed and selected from 286 submissions. They focus on both smart computing and communications fields and aimed to collect recent academic work to improve the research and practical application in the field.

Knowledge Science Engineering and Management

Knowledge Science  Engineering and Management

Download Knowledge Science Engineering and Management Book in PDF, Epub and Kindle

This book constitutes the refereed proceedings of the 9th International Conference on Knowledge Science, Engineering and Management, KSEM 2016, held in Passau, Germany, in October 2016. The 49 revised full papers presented together with 2 keynotes were carefully selected and reviewed from 116 submissions. The papers are organized in topical sections on Clustering and Classification; Text Mining and Lexical Analysis; Content and Document Analysis; Enterprise Knowledge; Formal Semantics and Fuzzy Logic; Knowledge Engineering; Knowledge Enrichment and Visualization; Knowledge Management; Knowledge Retrieval; Knowledge Systems and Security; Neural Networks and Artificial Intelligence; Ontologies; and Recommendation Algorithms and Systems.

Software Source Code

Software Source Code

Download Software Source Code Book in PDF, Epub and Kindle

This book will focus on utilizing statistical modelling of the software source code, in order to resolve issues associated with the software development processes. Writing and maintaining software source code is a costly business; software developers need to constantly rely on large existing code bases. Statistical modelling identifies the patterns in software artifacts and utilize them for predicting the possible issues.

Deep Learning for NLP and Speech Recognition

Deep Learning for NLP and Speech Recognition

Download Deep Learning for NLP and Speech Recognition Book in PDF, Epub and Kindle

This textbook explains Deep Learning Architecture, with applications to various NLP Tasks, including Document Classification, Machine Translation, Language Modeling, and Speech Recognition. With the widespread adoption of deep learning, natural language processing (NLP),and speech applications in many areas (including Finance, Healthcare, and Government) there is a growing need for one comprehensive resource that maps deep learning techniques to NLP and speech and provides insights into using the tools and libraries for real-world applications. Deep Learning for NLP and Speech Recognition explains recent deep learning methods applicable to NLP and speech, provides state-of-the-art approaches, and offers real-world case studies with code to provide hands-on experience. Many books focus on deep learning theory or deep learning for NLP-specific tasks while others are cookbooks for tools and libraries, but the constant flux of new algorithms, tools, frameworks, and libraries in a rapidly evolving landscape means that there are few available texts that offer the material in this book. The book is organized into three parts, aligning to different groups of readers and their expertise. The three parts are: Machine Learning, NLP, and Speech Introduction The first part has three chapters that introduce readers to the fields of NLP, speech recognition, deep learning and machine learning with basic theory and hands-on case studies using Python-based tools and libraries. Deep Learning Basics The five chapters in the second part introduce deep learning and various topics that are crucial for speech and text processing, including word embeddings, convolutional neural networks, recurrent neural networks and speech recognition basics. Theory, practical tips, state-of-the-art methods, experimentations and analysis in using the methods discussed in theory on real-world tasks. Advanced Deep Learning Techniques for Text and Speech The third part has five chapters that discuss the latest and cutting-edge research in the areas of deep learning that intersect with NLP and speech. Topics including attention mechanisms, memory augmented networks, transfer learning, multi-task learning, domain adaptation, reinforcement learning, and end-to-end deep learning for speech recognition are covered using case studies.

Proceeding of International Conference on Computational Science and Applications

Proceeding of International Conference on Computational Science and Applications

Download Proceeding of International Conference on Computational Science and Applications Book in PDF, Epub and Kindle

This book consists of high-quality papers presented at the International Conference on Computational Science and Applications (ICCSA 2021), held at Maharashtra Institute of Technology World Peace University, Pune, India, from 10 – 11 December 2021. It covers the latest innovations and developments in information and communication technology, discussing topics such as algorithms, data structures and applications; wireless and mobile networks; computer networks and communications; natural language processing and information theory; cryptography and information security.

Handbook of Green Computing and Blockchain Technologies

Handbook of Green Computing and Blockchain Technologies

Download Handbook of Green Computing and Blockchain Technologies Book in PDF, Epub and Kindle

This handbook provides a computational perspective on green computing and blockchain technologies. It presents not only how to identify challenges using a practical approach but also how to develop strategies for addressing industry challenges. Handbook of Green Computing and Blockchain Technologies takes a practical-oriented approach, including solved examples and highlights standardization, industry bodies, and initiatives. Case studies provide a deeper understanding of blockchain and are related to real-time scenarios. The handbook analyzes current research and development in green computing and blockchain analytics, studies existing related standards and technologies, and provides results on implementation, challenges, and issues in today’s society. FEATURES Analyzes current research developments in green computing and blockchain analytics Provides an analysis of implementation challenges and solutions Offers innovations in the decentralization process for the application of blockchain in areas such as healthcare, government services, agriculture, supply chain, financial, ecommerce, and more Discusses the impact of this technology on people’s lives, the way they work and learn, and highlights standardization, industry bodies, and initiatives This handbook will benefit researchers, software developers, and undergraduate and postgraduate students in industrial systems, manufacturing, information technology, computer science, manufacturing, communications, and electrical engineering.

掌桥科研:中文期刊与文献,核心期刊,一站式科研服务平台

A Study on the Relative Importance of Convolutional Neural Networks in Visually-Aware Recommender Systems

机译: 卷积神经网络在视觉知识推荐系统中的相对重要性研究