

FOLLOWUS
1.State Key Laboratory of Climate System Prediction and Risk Management/Key Laboratory of Meteorological Disaster, Ministry of Education (KLME), Joint International Research Laboratory of Climate and Environmental Change (ILCEC), Collaborative Innovation Center on Forecast and Evaluation of Meteorological Disasters (CIC-FEMD), Nanjing University of Information Science and Technology, Nanjing 210044, China
2.State Key Laboratory of Climate System Prediction and Risk Management/School of Marine Sciences, Nanjing University of Information Science and Technology, Nanjing 210044, China
3.Laoshan Laboratory, Qingdao 266237, China
rzhang@nuist.edu.cn
Received:06 December 2024,
Online First:10 February 2025,
Published:01 November 2025
Scan QR Code
ZHOU Lu,ZHANG Rong-Hua.The 3D-Geoformer for ENSO studies: a Transformer-based model with integrated gradient methods for enhanced explainability[J].Journal of Oceanology and Limnology,2025,43(06):1688-1708.
ZHOU Lu,ZHANG Rong-Hua.The 3D-Geoformer for ENSO studies: a Transformer-based model with integrated gradient methods for enhanced explainability[J].Journal of Oceanology and Limnology,2025,43(06):1688-1708. DOI: 10.1007/s00343-025-4330-y.
Deep learning (DL) has become a crucial technique for predicting the El Niño-Southern Oscillation (ENSO) and evaluating its predictability. While various DL-based models have been developed for ENSO predictions
many fail to capture the coherent multivariate evolution within the coupled ocean-atmosphere system of the tropical Pacific. To address this three-dimensional (3D) limitation and represent ENSO-related ocean-atmosphere interactions more accurately
a novel this 3D multivariate prediction model was proposed based on a Transformer architecture
which incorporates a spatiotemporal self-attention mechanism. This model
named 3D-Geoformer
offers several advantages
enabling accurate ENSO predictions up to one and a half years in advance. Furthermore
an integrated gradient method was introduced into the model to identify the sources of predictability for sea surface temperature (SST) variability in the eastern equatorial Pacific. Results reveal that the 3D-Geoformer effectively captures ENSO-related precursors during the evolution of ENSO events
particularly the thermocline feedback processes and ocean temperature anomaly pathways on and off the equator. By extending DL-based ENSO predictions from one-dimensional Niño time series to 3D multivariate fields
the 3D-Geoformer represents a significant advancement in ENSO prediction. This study provides details in the model formulation
analysis procedures
sensitivity experiments
and illustrative examples
offering practical guidance for the application of the model in ENSO research.
Amaya D J . 2019 . The Pacific meridional mode and ENSO: a review . Current Climate Change Reports , 5 ( 4 ): 296 - 307 , https://doi.org/10.1007/s40641-019-00142-x https://doi.org/10.1007/s40641-019-00142-x .
Barnston A G , Tippett M K , L'Heureux M L et al . 2012 . Skill of real-time seasonal ENSO model predictions during 2002-11: is our capability increasing? Bulletin of the American Meteorological Society , 93 ( 5 ): 631 - 651 , https://doi.org/10.1175/bams-d-11-00111.1 https://doi.org/10.1175/bams-d-11-00111.1 .
Behringer D , Xue Y . 2004 . Evaluation of the global ocean data assimilation system at NCEP: the Pacific Ocean . In: Proceedings of the Eighth Symposium on Integrated Observing and Assimilation Systems for Atmosphere, Oceans, and Land Surface, AMS 84 th Annual Meeting, Washington State Convention and Trade Center . Seattle, Washington .
Bi K F , Xie L X , Zhang H H et al . 2023 . Accurate medium-range global weather forecasting with 3D neural networks . Nature , 619 ( 7970 ): 533 - 538 , https://doi.org/10.1038/s41586-023-06185-3 https://doi.org/10.1038/s41586-023-06185-3 .
Bjerknes J . 1969 . Atmospheric teleconnections from the Equatorial Pacific . Monthly Weather Review , 97 ( 3 ): 163 - 172 , https://doi.org/10.1175/1520-0493(1969)097<0163:ATFTEP>2.3.CO;2. https://doi.org/10.1175/1520-0493(1969)097<0163:ATFTEP>2.3.CO;2.
Bonavita M . 2024 . On some limitations of current machine learning weather prediction models . Geophysical Research Letters , 51 ( 12 ): e2023 GL 107377 , https://doi.org/10.1029/2023gl107377 https://doi.org/10.1029/2023gl107377 .
Cachay S R , Erickson E , Bucker A F C et al . 2020 . Graph neural networks for improved El Niño forecasting . arXiv: 2012.01598 .
Callahan C W , Mankin J S . 2023 . Persistent effect of El Niño on global economic growth . Science , 380 ( 6649 ): 1064 - 1069 , https://doi.org/10.1126/science.adf2983 https://doi.org/10.1126/science.adf2983 .
Cao T W , Zheng F , Fang X H . 2024 . The asymmetry of air-sea coupled strength between the first-year and second-year La Niña events . Climate Dynamics , 62 ( 7 ): 6951 - 6963 , https://doi.org/10.1007/s00382-024-07259-2 https://doi.org/10.1007/s00382-024-07259-2 .
Cao X Q , Guo Y N , Liu B N et al . 2020 . ENSO prediction based on Long Short-Term Memory (LSTM) . IOP Conference Series: Materials Science and Engineering , 799 ( 1 ): 012035 , https://doi.org/10.1088/1757-899x/799/1/012035 https://doi.org/10.1088/1757-899x/799/1/012035 .
Carton J A , Giese B S . 2008 . A reanalysis of ocean climate using Simple Ocean Data Assimilation (SODA) . Monthly Weather Review , 136 ( 8 ): 2999 - 3017 , https://doi.org/10.1175/2007mwr1978.1 https://doi.org/10.1175/2007mwr1978.1 .
Chen H C , Tseng Y H , Hu Z Z et al . 2020 . Enhancing the ENSO Predictability beyond the Spring Barrier . Scientific Reports , 10 ( 1 ): 984 , https://doi.org/10.1038/s41598-020-57853-7 https://doi.org/10.1038/s41598-020-57853-7 .
Chen K , Han T , Gong J C et al . 2023a . FengWu: pushing the skillful global medium-range weather forecast beyond 10 days lead . arXiv: 2304.02948 .
Chen L , Zhong X H , Zhang F et al . 2023b . FuXi: a cascade machine learning forecasting system for 15-day global weather forecast . npj Climate and Atmospheric Science , 6 ( 1 ): 190 , https://doi.org/10.1038/s41612-023-00512-1 https://doi.org/10.1038/s41612-023-00512-1 .
Chen M N , Gao C , Zhang R-H . 2024a . How the central-western equatorial Pacific easterly wind in early 2022 affects the third-year La Niña occurrence . Climate Dynamics , 62 ( 5 ): 3047 - 3066 , https://doi.org/10.1007/s00382-023-07050-9 https://doi.org/10.1007/s00382-023-07050-9 .
Chen S F , Chen W , Yu B et al . 2023c . Enhanced impact of the Aleutian Low on increasing the Central Pacific ENSO in recent decades . npj Climate and Atmospheric Science , 6 ( 1 ): 29 , https://doi.org/10.1038/s41612-023-00350-1 https://doi.org/10.1038/s41612-023-00350-1 .
Chen S F , Chen W , Zhou W et al . 2024b . Interdecadal variation in the impact of Arctic sea ice on El Niño-southern oscillation: the role of atmospheric mean flow . Journal of Climate , 37 ( 21 ): 5483 - 5506 , https://doi.org/10.1175/JCLI-D-23-0733.1 https://doi.org/10.1175/JCLI-D-23-0733.1 .
Chiang J C H , Vimont D J . 2004 . Analogous Pacific and Atlantic meridional modes of tropical atmosphere-ocean variability . Journal of Climate , 17 ( 21 ): 4143 - 4158 , https://doi.org/10.1175/jcli4953.1 https://doi.org/10.1175/jcli4953.1 .
Ding R Q , Tseng Y H , Di Lorenzo E et al . 2022 . Multi-year El Niño events tied to the North Pacific Oscillation . Nature Communications , 13 ( 1 ): 3871 , https://doi.org/10.1038/s41467-022-31516-9 https://doi.org/10.1038/s41467-022-31516-9 .
Dong C M , Xu G J , Han G Q et al . 2022 . Recent developments in artificial intelligence in oceanography. Ocean - Land - Atmosphere Research , 2022 : 9870950 , https://doi.org/10.34133/2022/9870950 https://doi.org/10.34133/2022/9870950 .
Dong X , Zhang R-H , Hu J Y et al . 2025 . Thermodynamic processes prolong triple La Niña events in a hybrid coupled ocean-atmosphere model . Climate Dynamics , 63 ( 3 ): 92 , https://doi.org/10.1007/s00382-024-07564-w https://doi.org/10.1007/s00382-024-07564-w .
Du S Y , Zhang R-H . 2024 . U-Net models for representing wind stress anomalies over the tropical Pacific and their integrations with an intermediate coupled model for ENSO studies . Advances in Atmospheric Sciences , 41 ( 7 ): 1403 - 1416 , https://doi.org/10.1007/s00376-023-3179-2 https://doi.org/10.1007/s00376-023-3179-2 .
Eyring V , Bony S , Meehl G A et al . 2016 . Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization . Geoscientific Model Development , 9 ( 5 ): 1937 - 1958 , https://doi.org/10.5194/gmd-9-1937-2016 https://doi.org/10.5194/gmd-9-1937-2016 .
Fang X H , Zheng F , Li K X et al . 2023 . Will the historic southeasterly wind over the equatorial Pacific in March 2022 trigger a third-year La Niña event? Advances in Atmospheric Sciences , 40 ( 1 ): 6 - 13 , https://doi.org/10.1007/s00376-022-2147-6 https://doi.org/10.1007/s00376-022-2147-6 .
Feng L C , Zhang R-H , Yu B et al . 2020 . The roles of wind stress and subsurface cold water in the second-year cooling of the 2017 / 18 La Niña event. Advances in Atmospheric Sciences , 37 ( 16 ): 847 - 860 , https://doi.org/10.1007/s00376-020-0028-4 https://doi.org/10.1007/s00376-020-0028-4
Feng L C , Zhang R-H , Yu B et al . 2021 . On the second-year warming in late 2019 over the tropical Pacific and its triggering mechanism attributed to Indian Ocean effects . Advances in Atmospheric Sciences , 38 ( 12 ): 2153 - 2166 . https://doi.org/10.1007/s00376-021-1234-4 https://doi.org/10.1007/s00376-021-1234-4
Gao C , Chen M N , Zhou L et al . 2022 . The 2020-2021 prolonged La Niña evolution in the tropical Pacific . Science China Earth Sciences , 65 ( 12 ): 2248 - 2266 , https://doi.org/10.1007/s11430-022-9985-4 https://doi.org/10.1007/s11430-022-9985-4 .
Gao C , Zhang R-H . 2023 . A mechanism for the generation of a warm SST anomaly in the western equatorial Pacific: a pathway perspective . Journal of Geophysical Research: Oceans , 128 ( 11 ): e2023 JC 020119 , https://doi.org/10.1029/2023jc020119 https://doi.org/10.1029/2023jc020119 .
Gao C , Zhou L , Zhang R-H . 2023 . A transformer-based deep learning model for successful predictions of the 2021 second-year La Niña condition . Geophysical Research Letters , 50 ( 12 ): e2023 GL 104034 , https://doi.org/10.1029/2023gl104034 https://doi.org/10.1029/2023gl104034 .
Ham Y G , Kim J H , Kim E S et al . 2021 . Unified deep learning model for El Niño/Southern Oscillation forecasts by incorporating seasonality in climate data . Science Bulletin , 66 ( 13 ): 1358 - 1366 , https://doi.org/10.1016/j.scib.2021.03.009 https://doi.org/10.1016/j.scib.2021.03.009 .
Ham Y G , Kim J H , Luo J J . 2019 . Deep learning for multi-year ENSO forecasts . Nature , 573 ( 7775 ): 568 - 572 , https://doi.org/10.1038/s41586-019-1559-7 https://doi.org/10.1038/s41586-019-1559-7 .
Hu Z Z , Kumar A , Huang B H et al . 2020 . The interdecadal shift of ENSO properties in 1999/2000: a review . Journal of Climate , 33 ( 11 ): 4441 - 4462 , https://doi.org/10.1175/Jcli-D-19-0316.1 https://doi.org/10.1175/Jcli-D-19-0316.1 .
Huang A , Vega‐Westhoff B , Sriver R L . 2019 . Analyzing El Niño-Southern Oscillation predictability using long-short-term-memory models . Earth and Space Science , 6 ( 2 ): 212 - 221 , https://doi.org/10.1029/2018ea000423 https://doi.org/10.1029/2018ea000423 .
Kirtman B P , Min D , Infanti J M et al . 2014 . The North American multimodel ensemble: Phase-1 seasonal-to-interannual prediction; Phase-2 toward developing intraseasonal prediction . Bulletin of the American Meteorological Society , 95 : 585 - 601 , https://doi.org/10.1175/Bams-D-12-00050.1 https://doi.org/10.1175/Bams-D-12-00050.1 .
Larson S M , Kirtman B P . 2014 . The Pacific meridional mode as an ENSO precursor and predictor in the north American multimodel ensemble . Journal of Climate , 27 ( 18 ): 7018 - 7032 , https://doi.org/10.1175/Jcli-D-14-00055.1 https://doi.org/10.1175/Jcli-D-14-00055.1 .
Larson S M , Kirtman B P . 2015 . An alternate approach to ensemble ENSO forecast spread: application to the 2014 forecast . Geophysical Research Letters , 42 ( 21 ): 9411 - 9415 , https://doi.org/10.1002/2015gl066173 https://doi.org/10.1002/2015gl066173 .
Liang X F , Hu Z T , You X C . 2024 . ENSONet: a data-driven ENSO forecasting model with concise spatial location learning parameters and temporal embedding . Climate Dynamics , 62 ( 5 ): 4081 - 4098 , https://doi.org/10.1007/s00382-024-07119-z https://doi.org/10.1007/s00382-024-07119-z .
Liu T , Song X S , Tang Y M et al . 2022 . ENSO predictability over the past 137 years based on a CESM ensemble prediction system . Journal of Climate , 35 ( 2 ): 763 - 777 , https://doi.org/10.1175/Jcli-D-21-0450.1 https://doi.org/10.1175/Jcli-D-21-0450.1 .
Lu Y L , Feng J Q , Jia F et al . 2022 . Interdecadal change in the relationship between the El Niño-Southern Oscillation and the North/South Pacific meridional mode . Journal of Geophysical Research: Oceans , 127 ( 7 ): e2021 JC 018284 , https://doi.org/10.1029/2021jc018284 https://doi.org/10.1029/2021jc018284 .
Materia S , García L P , van Straaten C et al . 2024 . Artificial intelligence for climate prediction of extremes: state of the art, challenges, and future perspectives . WIREs Climate Change , 15 ( 6 ): e 914 , https://doi.org/10.1002/wcc.914 https://doi.org/10.1002/wcc.914 .
Mu B , Cui Y H , Yuan S J et al . 2024 . Incorporating heat budget dynamics in a Transformer-based deep learning model for skillful ENSO prediction . npj Climate and Atmospheric Science , 7 ( 1 ): 208 , https://doi.org/10.1038/s41612-024-00741-y https://doi.org/10.1038/s41612-024-00741-y .
Mu B , Qin B , Yuan S J . 2021 . ENSO-ASC 1.0.0: ENSO deep learning forecast model with a multivariate air-sea coupler . Geoscientific Model Development , 14 ( 11 ): 6977 - 6999 , https://doi.org/10.5194/gmd-14-6977-2021 https://doi.org/10.5194/gmd-14-6977-2021 .
Mu M , Qin B , Dai G K . 2025 . Predictability study of weather and climate events related to artificial intelligence models . Advances in Atmospheric Sciences , 42 ( 1 ): 1 - 8 , https://doi.org/10.1007/s00376-024-4372-7 https://doi.org/10.1007/s00376-024-4372-7 .
Qin B , Yang Z Y , Mu M et al . 2024 . The first kind of predictability problem of El Niño predictions in a multivariate coupled data-driven model . Quarterly Journal of the Royal Meteorological Society , 150 ( 765 ): 5452 - 5471 , https://doi.org/10.1002/qj.4882 https://doi.org/10.1002/qj.4882 .
Song C Y , Zhang X F , Zheng F et al . 2022 . The roles of off-equatorial subsurface cold-water incursions in triggering the second-year cooling of the La Niña event in 2021 . Journal of Marine Science and Engineering , 10 ( 11 ): 1667 , https://doi.org/10.3390/jmse10111667 https://doi.org/10.3390/jmse10111667 .
Sun M , Chen L , Li T et al . 2023 . CNN-based ENSO forecasts with a focus on SSTA zonal pattern and physical interpretation . Geophysical Research Letters , 50 ( 20 ): e2023 GL 105175 , https://doi.org/10.1029/2023gl105175 https://doi.org/10.1029/2023gl105175 .
Sundararajan M , Taly A , Yan Q Q . 2017 . Axiomatic attribution for deep networks . In: Proceedings of the 34th International Conference on Machine Learning . JMLR . org , Sydney, NSW, Australia . p. 3319 - 3328 .
Tang Y M , Zhang R-H , Liu T et al . 2018 . Progress in ENSO prediction and predictability study . National Science Review , 5 ( 6 ): 826 - 839 , https://doi.org/10.1093/nsr/nwy105 https://doi.org/10.1093/nsr/nwy105 .
Vaswani A , Shazeer N , Parmar N et al . 2017 . Attention is all you need . In: Proceedings of the 31st International Conference on Neural Information Processing Systems . Curran Associates Inc. , Long Beach, CA, USA . p. 6000 - 6010 .
Vimont D J , Wallace J M , Battisti D S . 2003 . The seasonal footprinting mechanism in the Pacific: implications for ENSO . Journal of Climate , 16 ( 16 ): 2668 - 2675 , https://doi.org/10.1175/1520-0442(2003)016<2668:TSFMIT>2.0.CO;2. https://doi.org/10.1175/1520-0442(2003)016<2668:TSFMIT>2.0.CO;2.
Wu X , Okumura Y M , Deser C et al . 2021 . Two-year dynamical predictions of ENSO event duration during 1954-2015 . Journal of Climate , 34 ( 10 ): 4069 - 4087 , https://doi.org/10.1175/jcli-d-20-0619.1 https://doi.org/10.1175/jcli-d-20-0619.1 .
Zhang R-H , Gao C . 2016 . The IOCAS intermediate coupled model (IOCAS ICM) and its real-time predictions of the 2015 - 16 El Niño event. Science Bulletin , 66 ( 13 ): 1061 - 1070 , https://doi.org/10.1007/s11434-016-1064-4 https://doi.org/10.1007/s11434-016-1064-4
Zhang R-H , Gao C , Feng L C . 2022 . Recent ENSO evolution and its real-time prediction challenges . National Science Review , 9 ( 4 ): nwac 052 , https://doi.org/10.1093/nsr/nwac052 https://doi.org/10.1093/nsr/nwac052 .
Zhang R-H , Rothstein L M , Busalacchi A J . 1998 . Origin of upper-ocean warming and El Niño change on decadal scales in the tropical Pacific Ocean . Nature , 391 ( 6670 ): 879 - 883 , https://doi.org/10.1038/36081 https://doi.org/10.1038/36081 .
Zhang R-H , Yu Y Q , Song Z Y et al . 2020 . A review of progress in coupled ocean-atmosphere model developments for ENSO studies in China . Journal of Oceanology and Limnology , 38 ( 4 ): 930 - 961 , https://doi.org/10.1007/s00343-020-0157-8 https://doi.org/10.1007/s00343-020-0157-8 .
Zhang R-H , Zhang W Z , Yu Y et al . 2025 . A hybrid coupled model for the tropical Pacific constructed by integrating ROMS with a statistical atmospheric model . Journal of Oceanology and Limnology , https://doi.org/10.1007/s00343-024-4194-6 https://doi.org/10.1007/s00343-024-4194-6 .
Zhang R-H , Zhou L , Gao C et al . 2024a . A transformer-based coupled ocean-atmosphere model for ENSO studies . Science Bulletin , 69 ( 15 ): 2323 - 2327 , https://doi.org/10.1016/j.scib.2024.04.048 https://doi.org/10.1016/j.scib.2024.04.048 .
Zhang R-H , Zhou L , Gao C et al . 2024b . Real-time predictions of the 2023-2024 climate conditions in the tropical Pacific using a purely data-driven Transformer model . Science China Earth Sciences , 67 ( 12 ): 3709 - 3726 , https://doi.org/10.1007/s11430-024-1396-x https://doi.org/10.1007/s11430-024-1396-x .
Zhao S , Jin F F , Stuecker M F et al . 2024 . Explainable El Niño predictability from climate mode interactions . Nature , 630 ( 8018 ): 891 - 898 , https://doi.org/10.1038/s41586-024-07534-6 https://doi.org/10.1038/s41586-024-07534-6 .
Zheng G , Li X F , Zhang R-H et al . 2020 . Purely satellite data-driven deep learning forecast of complicated tropical instability waves . Science Advances , 6 ( 29 ): eaba 1482 , https://doi.org/10.1126/sciadv.aba1482 https://doi.org/10.1126/sciadv.aba1482 .
Zheng Y Q , Chen S F , Chen W et al . 2023 . A continuing increase of the impact of the spring North Pacific meridional mode on the following winter El Niño and Southern Oscillation . Journal of Climate , 36 ( 2 ): 585 - 602 , https://doi.org/10.1175/JCLI-D-22-0190.1 https://doi.org/10.1175/JCLI-D-22-0190.1 .
Zhou L , Gao C , Zhang R-H . 2023 . A spatiotemporal 3D convolutional neural network model for ENSO predictions: a test case for the 2020/21 La Niña conditions . Atmospheric and Oceanic Science Letters , 16 ( 4 ): 100330 , https://doi.org/10.1016/j.aosl.2023.100330 https://doi.org/10.1016/j.aosl.2023.100330 .
Zhou L , Zhang R-H . 2022 . A hybrid neural network model for ENSO prediction in combination with principal oscillation pattern analyses . Advances in Atmospheric Sciences , 39 ( 6 ): 889 - 902 , https://doi.org/10.1007/s00376-021-1368-4 https://doi.org/10.1007/s00376-021-1368-4 .
Zhou L , Zhang R-H . 2023 . A self-attention-based neural network for three-dimensional multivariate modeling and its skillful ENSO predictions . Science Advances , 9 ( 10 ): eadf 2827 , https://doi.org/10.1126/sciadv.adf2827 https://doi.org/10.1126/sciadv.adf2827 .
Zhou L , Zhang R-H . 2024 . ENSO-related precursor pathways of interannual thermal anomalies identified using a transformer-based deep learning model in the tropical Pacific . Geophysical Research Letters , 51 ( 12 ): e2023 GL 107347 , https://doi.org/10.1029/2023gl107347 https://doi.org/10.1029/2023gl107347 .
Zhu J S , Wang W Q , Kumar A et al . 2024 . Assessment of a new global ocean reanalysis in ENSO predictions with NOAA UFS . Geophysical Research Letters , 51 ( 6 ): e2023 GL 106640 , https://doi.org/10.1029/2023gl106640 https://doi.org/10.1029/2023gl106640 .
Zhu Y C , Zhang R-H , Moum J N et al . 2022 . Physics-informed deep-learning parameterization of ocean vertical mixing improves climate simulations . National Science Review , 9 ( 8 ): nwac 044 , https://doi.org/10.1093/nsr/nwac044 https://doi.org/10.1093/nsr/nwac044 .
Zuo H , Balmaseda M A , Tietsche S et al . 2019 . The ECMWF operational ensemble reanalysis-analysis system for ocean and sea ice: a description of the system and assessment . Ocean Science , 15 ( 3 ): 779 - 808 , https://doi.org/10.5194/os-15-779-2019 https://doi.org/10.5194/os-15-779-2019 .
0
Views
18
Downloads
0
CSCD
Publicity Resources
Related Articles
Related Author
Related Institution
京公网安备11010802024621