Author: Iowa Climate Science Education

Claim: Google NeuralGCM AI Might Improve Climate Models

Essay by Eric Worrall

“… Google adds machine learning to climate models for ‘faster forecasts’ …”

The secret to better weather forecasts may be a dash of AI

Google adds machine learning to climate models for ‘faster forecasts’

Tobias Mann 
Sat 27 Jul 2024  // 13:27 UTC

…

In a paper published in the journal Nature this week, a team from Google and the European Centre for Medium-Range Weather Forecasts (ECMWF) detailed a novel approach that uses machine learning to overcome limitations in existing climate models and try to generate forecasts faster and more accurately than existing methods.

Dubbed NeuralGCM, the model was developed using historical weather data gathered by ECMWF, and uses neural networks to augment more traditional HPC-style physics simulations.

As Stephan Hoyer, one of the crew behind NeuralGCM wrote in a recent report, most climate models today make predictions by dividing up the globe into cubes 50-100 kilometers on each side and then simulating how air and moisture move within them based on known laws of physics.

NeuralGCM works in a similar fashion, but the added machine learning is used to track climate processes that aren’t necessarily as well understood or which take place at smaller scales.

…

Read more: https://www.theregister.com/2024/07/27/google_ai_weather/

The abstract of the study;

Neural general circulation models for weather and climate

Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Sanchez-Gonzalez, Matthew Willson, Michael P. Brenner & Stephan Hoyer 

Abstract

General circulation models (GCMs) are the foundation of weather and climate prediction1,2. GCMs are physics-based simulators that combine a numerical solver for large-scale dynamics with tuned representations for small-scale processes such as cloud formation. Recently, machine-learning models trained on reanalysis data have achieved comparable or better skill than GCMs for deterministic weather forecasting3,4. However, these models have not demonstrated improved ensemble forecasts, or shown sufficient stability for long-term weather and climate simulations. Here we present a GCM that combines a differentiable solver for atmospheric dynamics with machine-learning components and show that it can generate forecasts of deterministic weather, ensemble weather and climate on par with the best machine-learning and physics-based methods. NeuralGCM is competitive with machine-learning models for one- to ten-day forecasts, and with the European Centre for Medium-Range Weather Forecasts ensemble prediction for one- to fifteen-day forecasts. With prescribed sea surface temperature, NeuralGCM can accurately track climate metrics for multiple decades, and climate forecasts with 140-kilometre resolution show emergent phenomena such as realistic frequency and trajectories of tropical cyclones. For both weather and climate, our approach offers orders of magnitude computational savings over conventional GCMs, although our model does not extrapolate to substantially different future climates. Our results show that end-to-end deep learning is compatible with tasks performed by conventional GCMs and can enhance the large-scale physical simulations that are essential for understanding and predicting the Earth system.

Read more: https://www.nature.com/articles/s41586-024-07744-y

Reading the main study, they appear to be claiming adding neural net magic sauce produces better short term weather predictions and climate predictions.

The researchers attempted to test their model by holding back some of the training data, and using their trained neural network to product weather forecasts based on real world data which had never been seen before by the AI. They also discuss how their model predictions diverge from reality after 3 days – “At longer lead times, RMSE rapidly increases owing to chaotic divergence of nearby weather trajectories, making RMSE less informative for deterministic models”, but claim their approach still performs better than traditional approaches, once that chaotic divergence is taken into account.

I’m a bit dubious about putting faith in the predictive skill of neural net black boxes. History is littered with scientists who followed all the steps the authors described, only to see the neural net diverge wildly from expected behaviour on demonstration day. I would have preferred if they made more effort to reverse engineer their neural net, to tease out what it actually discovered, if anything, to see if it discovered new atmospheric physics which can be used to create better deterministic white box models.

In 2018 Amazon suffered a serious embarrassment when one of their neural nets went off the rails. They tried to use neural nets to filter tech candidates, but they discovered the neural net exhibited bias against women candidates. The neural net had noticed most of the candidates were men, and inferred it should discard applications from female candidates based on their gender.

Anyone who thinks there is any basis to that Amazon neural net gender bias needs to pay a visit to a tech shop in Asia. Somehow in the West we are convincing our girls from a young age they are not suited for tech jobs. The few women who make it through this Western cultural filter, women like Margaret Hamilton who led the team which programmed the Apollo guidance computer, more than demonstrate their ability. Hamilton is credited with inventing the term “software engineering”.

Neural Nets are incredibly useful, they can identify relationships which aren’t obvious. But that ability to see non-obvious patterns in data carries a severe risk of false positives, seeing patterns which don’t exist. Especially when analysing physical phenomena, if you don’t try to reverse engineer the inner workings of your magic black box neural network once it has allegedly demonstrated superior skill, you just don’t know whether what you are seeing is genuine skill or a subtle false positive.

via Watts Up With That?

https://ift.tt/LV4lwkq

July 28, 2024 at 08:03PM

Biggest flare of Solar Cycle 25

Major Farside Solar Flare

Spaceweather concludes: ‘The source of this blast will rotate around to face our planet a week to 10 days from now, so stay tuned.’

via Tallbloke’s Talkshop

https://ift.tt/zeMNDaO

July 28, 2024 at 04:13PM

When Green Dreams Go Up in Smoke: The Great Lithium Truck Fiasco of I-15

Once upon a time in the magical land of California, a truck carrying the very essence of green dreams – lithium-ion batteries – decided to take a nap on the side of I-15. This wasn’t just any ordinary nap, mind you; it was a fiery, apocalyptic slumber that would go down in the annals of transportation history.

Now, let’s not forget that lithium-ion batteries are the darlings of the green movement. They’re supposed to be the answer to all our energy woes, powering everything from our electric cars to our smartphones. But, as the saying goes, “with great power comes great responsibility,” and in this case, the responsibility to not spontaneously combust on a major highway.

As the truck’s cargo decided to go full-on barbecue, the brave firefighters of San Bernardino County rushed to the scene, only to find themselves scratching their heads. You see, lithium-ion fires are a bit like that one friend who refuses to leave the party; they just keep going and going and going. Water is out of the question, as it only makes things worse. So, the firefighters had to stand back and watch the spectacle unfold, hoping that the batteries would eventually burn themselves out.

Meanwhile, the poor souls who found themselves stuck on I-15 were left to ponder the irony of it all. Here they were, trying to do their part for the environment by not driving gas-guzzling SUVs, only to be trapped in a traffic apocalypse from hell, courtesy of the very technology that was supposed to save us from ourselves.

The situation was so dire that some motorists were forced to spend the night in their cars, dreaming of the days when a simple gas-powered engine could get them from point A to point B without the risk of spontaneous combustion. It’s almost as if Mother Nature herself was sending a message: “You want green? I’ll give you green… the color of a fire truck.”

In the end, after 44 hours of chaos and confusion, the northbound lanes of I-15 were finally reopened. The charred remains of the lithium-ion batteries were a stark reminder that, while the road to a greener future is paved with good intentions, it’s also littered with the smoldering remnants of our best-laid plans.

So, the next time you’re stuck in traffic, dreaming of a world powered by clean, renewable energy, just remember: it’s not easy being green. And maybe, just maybe, we should take a step back and reevaluate our priorities before we put all our eggs in one lithium-ion basket.

via Watts Up With That?

https://ift.tt/OptgWD7

July 28, 2024 at 04:02PM

Crowdstrike conducted malware gain of function

A cybersecurity company purporting to protect its customers against malware must have a database of known malware. It is one of the company’s most important assets. When CrowdStrike was founded in 2012, it lacked a database of its own. Its main asset was Shawn Henry, hired from the position of Executive Assistant to the FBI Director. Instead of creating or renting malware signatures from another vendor, CrowdStrike deployed a tool called CrowdRE, short for Crowd Reverse Engineering. This was a malware repository with a reverse engineering tool. Officially, it was intended for security researchers to analyze malware. However, it also allowed malware authors worldwide to benefit from public reverse engineering and borrow modules from this repository for their own malware. Anyone with a Google account was welcome to join CrowdRE.

Many legitimate security researchers have contributed their malware samples. That allowed CrowdStrike to acquire many malware signatures. CrowdRE also had the effect of proliferating malware, known to Crowdstrike. Despite this, CrowdStrike‘s database was small compared to its competitors, leading the company to specialize in baseless attribution. Initially, CrowdStrike was likely more an aid to malware developers than a defender against them. CrowdStrike was aware that malware authors often borrow code from each other, even without the original author’s knowledge, and that their attribution model was incorrect.

See CrowdStrike Streamlines Malware Reverse Engineering With CrowdRE (SecurityWeek), Crowdstrike blog post, CrowdRE tutorial video (archived page), and a CrowdRE slide show, all from 2012.

First published on X.

via Science Defies Politics

https://ift.tt/U9yA2lO

July 28, 2024 at 12:22PM