Latest In


Deep Learning In Electron Microscopy – Overview Of Its Applications

The application of deep learning in electron microscopy is increasing, resulting in enormous data sets that cannot be analyzed using manually built methods.

Author:Suleman Shah
Reviewer:Han Ju
Oct 03, 20221 Shares156 Views
The application of deep learning in electron microscopyis increasing, resulting in enormous data sets that cannot be analyzed using manually built methods.
Deep learning has enabled superhuman performance in picture categorization, medical analysis, voice recognition, and various other applications.
It relieves physicists of the necessity to create equations to represent complex processes. Because modern artificial neural networks (ANNs) include millions of parameters, inference on graphics processing units (GPUs) or other hardware accelerators sometimes takes tens of milliseconds.
They use their grasp of physics to speed up time-consuming computations and increase procedure accuracy.

Applications Of Deep Learning In Electron Microscopy

Improving Signal-To-Noise

Deep learning is often used to increase the signal-to-noise ratio. Many classic denoising methods aren't based on deep learning. To denoise signals, they use an increasingly precise model of physics.
Traditional algorithms, on the other hand, are constrained by the difficulty of programmatically defining a complex world. Non-statistical noise is often reduced via hardware.
Most electron microscopy techniques teach artificial neural networks to map low-quality experimental pictures to corresponding high-quality observations.
Until yet, most artificial neural networks that enhance electron microscope signal-to-noise have been trained to reduce statistical noise.
Other methods for correcting electron microscope scan aberrations and specimen drift have been devised.

Compressed Sensing

The effective reconstruction of a signal from a subset of observations is known as compressed sensing. Compressed sensing in scanning transmission electron microscopy has reduced electron beam exposure and scan time by 10-100 with little information loss.
Upsampling or infilling a regularly spaced grid of signals is the most common way of compressed sensing. Deep learning may use physics knowledge to infill pictures and improve scanning electron microscope (SEM) resolution.
Sparse data gathering approaches that work well with traditional scanning transmission electron microscopy electron beam deflection devices have also been examined.
Spirals with constant angular velocity put the least strain on electron beam deflectors, but they are prone to systematic picture distortions owing to deflector reaction delays.
Fixed dosages are ideal because they facilitate visual examination and reduce scanning transmission electron microscopy noise's dose dependency.


Since convolutional neural networks (CNNs) achieved a breakthrough in classification accuracy on ImageNet, deep learning has been the foundation of cutting-edge classification. Most classifiers are single feedforward neural networks (FNNs) trained to predict discrete labels.
Electron microscopy applications include categorizing image area quality, material structures, and image resolution. However, siamese and dynamically parameterized networks may learn to recognize pictures faster.
Finally, labeling artificial neural networks may be trained to predict continuous characteristics like mechanical qualities.
Labeling artificial neural networks are often used in conjunction with other approaches. Artificial neural networks, for example, may be used to identify particle locations, making further processing easier.

Semantic Segmentation

The categorization of pixels into distinct categories is known as semantic segmentation. Electron microscopy applications include the automated detection of local characteristics in nanoparticles such as defects, dopants, material phases, material structures, dynamic surface phenomena, and chemical phases.
Simple criteria were utilized in early efforts to semantic segmentation. However, such approaches were not resistant to a wide range of data.
Following that, other adaptive algorithms based on soft computing and fuzzy algorithms that used geometric forms as priors were created. However, these approaches were constrained by pre-programmed characteristics and struggled to handle a wide range of input.
Deep neural networks have been taught to segment pictures to increase performance semantically. Semantic segmentation deep neural networks have been created for focused ion beam scanning electron microscopy, scanning electron microscope, scanning transmission electron microscopy, and transmission electron microscopy.
Outside of electron microscopy, deep learning-based semantic segmentation offers a wide range of applications, including autonomous driving, nutritional monitoring, magnetic resonance pictures, medical images such as prenatal ultrasound, and satellite image translation.
Most deep neural networks for semantic segmentation are trained using human-segmented photos. Human labeling, however, may be excessively costly, time-consuming, or unsuitable for sensitive data.
Unsupervised semantic segmentation may circumvent these challenges by learning to segment pictures using an extra dataset of segmented images or image-level labels. On the other hand, unsupervised semantic segmentation networks are often less accurate than supervised networks.

Exit Wavefunction Reconstruction

Exiting materials use electron wavefunctions for determining projected potentials and corresponding crystal structure information, information storage, point spread function deconvolution, improving contrast, aberration correction, thickness measurement, and determining electric and magnetic structure.
Exit wavefunctions are often rebuilt iteratively from focussed series or captured using electron holography. On the other hand, Iterative reconstruction is often too slow for live applications, while holography is sensitive to distortions and may need costly microscope modifications.
Non-iterative deep neural network-based approaches have been devised for reconstructing optical exit wavefunctions from focused series or single pictures. Deep learning is being used more and more in accelerated quantum physics.
Non-iterative approaches have also been developed for recovering phase information from single photos that do not rely on artificial neural networks.
However, in the Fresnel regime, they are confined to defocused pictures, while in the Fraunhofer regime, they are limited to non-planar incident wavefunctions.
Combination of the circuit in head shape
Combination of the circuit in head shape

Optimization Of Deep Learning Models

Machine learning system training, testing, deployment, and maintenance are time-consuming and costly. Typically, the initial step is to prepare training data and set up data pipelines for artificial neural networks training and assessment.
Artificial neural network parameters are often randomly initialized for gradient descent optimization, maybe as part of an automated machine learning (autoML) process. Reinforcement learning is an optimization problem in which the loss represents a discounted future reward.
Artificial neural network components are often regularized during training to stabilize, expedite convergence, or enhance performance. Finally, learned models may be optimized for rapid deployment.

Gradient Descent

Gradient descent is used to train most artificial neural networks iteratively. Intermediate stages of forwarding propagation are often held in memory to reduce computation. Backpropagation memoization is enabled by successively calculating gradients with respect to trainable parameters.
However, gradient descent is not an appropriate model of biological learning in general. Gradient descent works well in the high-dimensional optimization spaces of overparameterized artificial neural networks because the likelihood of being caught in suboptimal local minima lowers as the number of dimensions increases.
The most basic optimizer is 'vanilla' stochastic gradient descent (SGD), in which a trainable parameter perturbation is the product of a learning rate and a loss derivative. Many optimizers include a momentum component that weighs an average of gradients with previous gradients to speed up convergence.
Adaptive optimizers may be coupled with adaptive learning rate or gradient clipping to avoid learning from being destabilized by spikes in gradient sizes.
By splitting gradients by their predicted sizes, adaptive optimizers mitigate disappearing and bursting gradients. Because their maximum gradient is 1/4, deep neural networks with logistic sigmoid activations often display vanishing gradients.
Theoretically, stepwise exponentially degraded learning rates are often ideal. Simulated annealing may be used in conjunction with gradient descent to boost performance. Other technologies competing with deep reinforcement learning include evolutionary and genetic algorithms.

Reinforcement Learning

A machine learning system, or 'actor,' is taught to execute a series of actions in reinforcement learning. Applications include self-driving cars, network control, energy, environmental management, gaming, and robotic manipulation.
To optimize an MDP, a discounted future reward, qt, is often derived using Bellman's equation from step rewards. For continuous control tasks optimized by deep deterministic policy gradients, adding Ornstein-Uhlenbeck noise to actions is effective. Other exploratory tactics involve paying actors for improving action entropy and intrinsic motivation.
Many algorithms work in an intermediate mode, in which data acquired by an online policy is saved in an experience replay buffer for offline learning.
Prioritizing the replay of data with significant losses or data that produces substantial policy improvements, on the other hand, often increases actor performance.

Automatic Machine Learning

Several AutoML methods are available for creating and optimizing artificial neural network topologies and learning policies for a dataset of input and target output pairs. The majority of AutoML algorithms are built on reinforcement learning or evolutionary algorithms.
AdaNet, Auto-DeepLab, AutoGAN, Auto-Keras, auto-sklearn, DARTS+, EvoCNN, H2O, Ludwig, MENNDL, NASBOT, XNAS, and other AutoML algorithms are examples.
AutoML is gaining popularity because it outperforms human developers and allows human developer time to be exchanged for possibly cheaper computer time. AutoML is now constrained to pre-existing artificial neural network designs and learning rules.


Multiplicative weights or additive biases are examples of trainable parameters. Initializing parameters with too little or too high values might result in delayed learning or divergence.
Careful initialization may help prevent gradient descent training from becoming unstable due to disappearing or bursting gradients, or a significant variety of length scales between layers.
Recurrent neural networks primarily use some initializerscreated. Orthogonal initialization, for example, often enhances recurrent neural network training by lowering sensitivity to disappearing and exploding gradients.
Biases are typically started with zeros in most artificial neural networks. However, long short-term memory forgets gates are often created with ones to reduce forgetting at the start of training.


Many regularization strategies adjust learning algorithms to increase artificial neural network performance. Most deep neural network optimization uses L2 regularization because subtracting its gradient is computationally efficient.
LX regularization is most effective at the beginning of training and becomes less relevant as training progresses toward convergence. Gradient clipping is a technique that speeds up learning by minimizing huge gradients and is most typically used with recurrent neural networks.
Dropout often decreases overfitting by utilizing just a subset of layer I output during training and multiplying all outputs by pi for inference.
Dropout is often surpassed by Shakeout, a dropout modification that randomly amplifies or reverses output contributions to the following layer. Many regularization strategies make use of supplementary training data.

Data Pipeline

Data preparation is often parallelized over many CPU cores in efficient pipelines. Massive datasets may be kept in RAM to reduce data access times, while large dataset pieces are often fetched from files.
During gradient descent training, batch data may be randomly sampled with replacement. Most current deep learning frameworks provide efficient and simple capabilities for controlling data sampling.

Model Evaluation

Most artificial neural networks are assessed using 1-fold validation, which divides a dataset into training, validation, and test sets. Following artificial neural network optimization using a training set, generalization ability is tested with a validation set.
Several validations may be conducted for training with early stopping or architecture selection. Increasing the size of the training set typically improves model accuracy, whereas increasing the size of the validation set reduces performance uncertainty.


If an artificial neural network is used on numerous devices, such as electron microscopes, a distinct model for each device may be trained to reduce training needs.
Most artificial neural networks generated by researchers were not implemented in 2020; however, deployment will become a more important factor as deep learning's position in electron microscopy grows.
Most artificial neural networks, such as MobileNets, are optimized for inference by reducing parameters and operations during training, but less important operations may also be pruned after training.


There are many techniques to explain artificial intelligence(XAI). Saliency is a primary method for XAI in which gradients of outputs concerning inputs correspond with their relevance.
Some electron microscopists are hesitant to engage with artificial neural networks owing to a lack of interpretability.

People Also Ask

Is Deep Learning Used In Image Recognition?

The emergence of deep learning in conjunction with powerful AI technologyand graphical processing units allowed significant advances in picture identification.
Deep learning enables picture classification and facial recognition algorithms to outperform humans in termsof performance and real-time item identification.

What Is Deep Learning For Perception?

Deep learning is an artificial intelligence method that trains deep artificial neural networks to solve challenging tasks.

What Is The Concept Of Deep Learning?

Deep learning is a subset of machine learning that is a three or more-layered neural network. These neural networks seek to imitate the activity of the human brain, although with limited success, enabling it to "learn" from enormous volumes of data.

How Is Deep Learning Used In Image Processing?

Deep learning uses neural networks to learn useful representations of features directly from data. For example, you can use a pre-trained neural network to identify and remove artifacts like noise from images.


The essential component of deep learning in electron microscopy is that it introduces new difficulties that may lead to machine learning advancements. CIFAR-10 and MNIST are simple benchmarks that have been solved.
Following that, increasingly demanding benchmarks such as Fashion-MNIST were established. However, since they do not introduce fundamentally new difficulties, they only partly address concerns with solved datasets.
In contrast, new issues often need new solutions. The problem is to train a vast model for high-resolution photos, but training is unstable if tiny batches are used to fit it into graphical processing units' memory. Similar issues exist, and machine learning and electron microscopy advancements are possible.
Jump to
Suleman Shah

Suleman Shah

Suleman Shah is a researcher and freelance writer. As a researcher, he has worked with MNS University of Agriculture, Multan (Pakistan) and Texas A & M University (USA). He regularly writes science articles and blogs for science news website and open access publishers OA Publishing London and Scientific Times. He loves to keep himself updated on scientific developments and convert these developments into everyday language to update the readers about the developments in the scientific era. His primary research focus is Plant sciences, and he contributed to this field by publishing his research in scientific journals and presenting his work at many Conferences. Shah graduated from the University of Agriculture Faisalabad (Pakistan) and started his professional carrier with Jaffer Agro Services and later with the Agriculture Department of the Government of Pakistan. His research interest compelled and attracted him to proceed with his carrier in Plant sciences research. So, he started his Ph.D. in Soil Science at MNS University of Agriculture Multan (Pakistan). Later, he started working as a visiting scholar with Texas A&M University (USA). Shah’s experience with big Open Excess publishers like Springers, Frontiers, MDPI, etc., testified to his belief in Open Access as a barrier-removing mechanism between researchers and the readers of their research. Shah believes that Open Access is revolutionizing the publication process and benefitting research in all fields.
Han Ju

Han Ju

Hello! I'm Han Ju, the heart behind World Wide Journals. My life is a unique tapestry woven from the threads of news, spirituality, and science, enriched by melodies from my guitar. Raised amidst tales of the ancient and the arcane, I developed a keen eye for the stories that truly matter. Through my work, I seek to bridge the seen with the unseen, marrying the rigor of science with the depth of spirituality. Each article at World Wide Journals is a piece of this ongoing quest, blending analysis with personal reflection. Whether exploring quantum frontiers or strumming chords under the stars, my aim is to inspire and provoke thought, inviting you into a world where every discovery is a note in the grand symphony of existence. Welcome aboard this journey of insight and exploration, where curiosity leads and music guides.
Latest Articles
Popular Articles