GraphEM: Extracting Order Descriptors Through Multi-Modal Graph Analytics

Describing order in materials is a critical step in achieving performance for applications such as energy storage, computing, and sensing. However, order is encoded across multiple data modalities, which are often challenging for humans to interrogate and interpret. Graph-based segmentation overcomes these challenges by representing each pixel in an image as a node in a graph, where the edges of the graph are weighted by the similarity between pixels. The graph is then partitioned into segments using a variety of algorithms. This approach is well-suited for the analysis of complex materials microstructures, particularly when multiple imaging modalities are used. We used a combination of high-angle annular dark-field (HAADF) scanning transmission electron microscopy (STEM) and energy dispersive X-ray spectroscopy (EDS) to characterize the microstructure of a complex material. The HAADF images provided information about the morphology of the material while the EDS data provided information about the chemical composition. By combining these two modalities, we were able to segment the images into different phases and identify the chemical composition of each phase. This information was then used to train a machine learning model to predict the properties of the material.

More Information

• Ter-Petrosyan, A., Holden, M., Bilbrey, J.A., Akers, S., Doty, C., Yano, K.H., Wang, L., Paudel, R., Lang, E., Hattar, K., Comes, R.B., Du, Y., Matthews, B.E., and S.R. Spurgeon. “Revealing the evolution of order in materials microstructures using multi-modal computer vision.” Arxiv Preprint (2024). [DOI:10.48550/arXiv.2411.09896]

• To download the code, visit: https://github.com/pnnl/GraphEM

EELSTM: Forecasting of In Situ Data

Reward-based decision-making is directly linked to our ability to accurately forecast, or anticipate, changes in a system or process. Effective forecasting is essential for many disciplines and technologies we take for granted, ranging from meteorology to the power grid and from stock trading to logistics. The recent rise of autonomous vehicles, including automobiles, drones, and spacecraft, has been propelled by advanced forecasting models deployed on high performance computing platforms. Abundant low-cost computing and the proliferation of machine learning (ML) have enabled many new real-time forecasting approaches. When performed correctly, forecasting can save time, reduce cost, and guide scientific discovery by helping direct decision-making. We have developed deep learning-based approaches to forecasting of in situ imaging and electron energy loss spectroscopy data, allowing us to anticipate and respond to changes in materials and chemical systems for self-driving experimentation.

Students performing this research were supported by the UW Industry Capstone Program and the National Science Foundation.

More Information

• Lewis, N., Jin, Y., Tang, X., Shah, V., Doty, C., Matthews, B.E., Akers, S. and S.R. Spurgeon. “Forecasting of in situ electron energy loss spectroscopy.” npj Computational Materials. 8 (2022): 252. DOI:10.1038/s41524-022-00940-2 [Download PDF]

• Fu, W., Spurgeon, S.R., Wang, C., Shao, Y., Wang, W. and A. Peles. “Deep-learning-based prediction of nanoparticle phase transitions during in situ transmission electron microscopy.” Arxiv Preprint. https://arxiv.org/abs/2205.11407

• To download the code, visit: https://github.com/pnnl/EELSTM

pyCHIP: A Graphical User Interface for Few-Shot Machine Learning-Based Segmentation

The recent growth in data volumes produced by modern electron microscopes requires rapid, scalable, and flexible approaches to image segmentation and analysis. Few-shot machine learning, which can richly classify images from a handful of user-provided examples, is a promising route to high-throughput analysis. However, current command-line implementations of such approaches can be slow and unintuitive to use, lacking the real-time feedback necessary to perform effective classification. pyCHIP is a Python-based graphical user interface that enables end users to easily conduct and visualize the output of few-shot learning models. This interface is lightweight and can be hosted locally or on the web, providing the opportunity to reproducibly conduct, share, and crowd-source few-shot analyses.

Students performing this research were supported by the UW-DIRECT Data Science Capstone Program and the National Science Foundation.

More Information

• Doty, C., Gallagher, S., Cui, W., Chen, W., Bhushan, S., Oostrom, M., Akers, S., and S.R. Spurgeon. “Design of a graphical user interface for few-shot machine learning-based classification of electron microscopy data.” Computational Materials Science. 203.15 (2021): 111121. DOI:10.1016/j.commatsci.2021.111121 [Download PDF]

• Akers, S., Kautz, E., Trevino-Gavito, A., Olszta, M., Matthews, B., Wang, L., Du, Y., and S.R. Spurgeon. “Rapid and flexible segmentation of electron microscopy data using few-shot machine learning.“ npj Computational Materials. 7 (2021): 187. DOI:10.1038/s41524-021-00652-z [Download PDF]

• To download the code, visit: https://github.com/pnnl/pychip_gui

TEMWizard: A Graphical Frontend for Atomap Image Analysis

Extraction of quantitative crystallographic information from atomic-resolution transmission electron microscopy (TEM) images is challenging, since data are often noisy and artifacted. The Atomap package permits fitting and analysis of key features in atomic-resolution data, but its command-line format does not lend itself to facile and dynamic exploration of image data. TEMWizard provides an easy-to-use graphical user interface (GUI) for Atomap, allowing the user to provide correct inputs and visualize lattice displacements for analysis of phenomena such as octahedral rotations or bond distortions.

More Information

• Wang, L., Zhao, J., Kuo, C-T., Matthews, B.E., Oostrom, M.T., Spurgeon, S.R., Bowden, M.E., Lee, S-J., Lee, J-S., Guo, E-J., Wang, J., Chambers, S.A., and Y. Du. “Synthesis and electronic properties of epitaxial SrNiO3/SrTiO3 superlattices.” Physical Review Materials. 6 (2022): 075006. DOI:10.1103/PhysRevMaterials.6.075006

• To download the code, visit: https://github.com/pnnl/temwizard

• To learn more about Atomap, visit: https://atomap.org/

xtl-converter: Data Preparation for µSTEM TEM Image Simulation

Image simulation is an important part of quantitative analysis of electron microscopy data. The µSTEM package (https://github.com/HamishGBrown/MuSTEM) allows for rigorous simulation of images and ionization maps to aid in this process. However, the preparation of data from crystal files requires extensive reformatting and can be time-consuming. The xtl-converter code aids in this process by reformatting VESTA fractional coordinates (*.xtl) files into the proper configuration.

More Information

• Spurgeon, S.R., Du, Y., and S.A. Chambers. “Measurement error in atomic-scale scanning transmission electron microscopy–energy-dispersive X-ray spectroscopy (STEM-EDS) mapping of a model oxide interface.” Microscopy and Microanalysis. 23.3 (2017): 513–517. DOI:10.1017/S1431927617000368 [Download PDF]

• To download the code, visit: https://github.com/stevenspurgeon/xtl-converter