| Author | Message |
|---|
|
|
|
Continuing the thread from the php board.
Here are another two runs with the same parameters but very different results.
<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=438879&phase=AT#graph">here</a> and <a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=438878&phase=AT#graph">here</a>
Again it seems that there is a cold equator on the second run, though the climate sensitivity seems to be similar.
____________
____________________________<br>
<a href="http://www.boincforum.info/boinc/">boinc forum</a> and <a href="http://www.uk4cp.co.uk/">United Kindom</a> team, my climate change <a href="http://www.livejournal.com/users/mike_atkinson/">blog</a>. |
|
|
|
|
|
Unusual to see a cold equator where critical relative humidity gets as low as 0.6. However both seem to have been produced by a reliable computers.
I find this more worrying than other pairs seen.
I am not sure how you calculate climate sensitivity to be similar. Isn't it the difference between the equilibrium level that would be reached if phases 2 and 3 were continued?
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
> Unusual to see a cold equator where critical relative humidity gets as low as
> 0.6. However both seem to have been produced by a reliable computers.
>
> I find this more worrying than other pairs seen.
>
> I am not sure how you calculate climate sensitivity to be similar. Isn't it
> the difference between the equilibrium level that would be reached if phases 2
> and 3 were continued?
>
Looking at the graphs again I find I was wrong. I thought that the second run had a phase 3 that showed signs of a cold equator but at about 3 degrees above the phase 2 value. 3 degrees is the climate sensitivity of the first run and so they could be said to be similar. However, phase 3 of the second run seems to be stable and reach equilibrium after about a 1.5 degrees rise in temperature.
I agree it looks worrying. If the reasons for this are not contained in the papers about to be published then perhaps a technical note could be placed on this site explaining what is going on. There are several unexplained things happening with the experiment:
1. the same workunit on different computers seems to be giving different weather. This is true even if exactly the same download is used as per the threads on the php board.
2. the same workunit may give both stable and unstable phase 2. Probably caused by some weather conditions in the unstable case leading to cold equators, but this would be nice to be confirmed.
3. there are hints that the runs in this thread have different climate sensitivities for the same workunit (though this is difficult to verify as the second run has an unstable phase 2). Is this the case and if so what is the cause?
4. Are all unstable runs grouped in the parameter space, or are some scattered thoughout the parameter space? Is there a difference between those that are always unstable and those that are just sometimes unstable?
5. climate sensitivity seems to be usually measured as the difference between the average of the control (phase 2) and the 2xCO2 (phase 3), averaged over 10 years after equilibrium has been reached. Many of the phase 3 temperatures are still increasing, and most only reach equilibrium a few years before the end of phase 3, so how is climate sensitivity going to be measured? A result from experiment 1 + experiment 2 could be a pdf of the climate sensitivity for HadSM3 (weighted by the hindcast of experiment 2 using HadCM3), but the extremes (which cpdn was designed to cover) won't be represented if they are not allowed to reach equilibrium.
6. Given the same workunit giving different weather on different PCs is there any need for an initial condition ensemble?
____________
____________________________<br>
<a href="http://www.boincforum.info/boinc/">boinc forum</a> and <a href="http://www.uk4cp.co.uk/">United Kindom</a> team, my climate change <a href="http://www.livejournal.com/users/mike_atkinson/">blog</a>. |
|
|
|
|
|
Two flavors from the same Model, both run on P4's but with different versions of Windoze. They're similar but far from identical.
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/workunit.php?wuid=212401
(HostID #4741 is mine.)
____________
Greetings from coastal Washington state, the scenic US Pacific Northwest.
Important stuff no longer here: http://www.climateprediction.net/board/viewforum.php?f=44 |
|
|
|
|
|
(Accidental duplication, sorry.)
____________
Greetings from coastal Washington state, the scenic US Pacific Northwest.
Important stuff no longer here: http://www.climateprediction.net/board/viewforum.php?f=44 |
|
|
|
|
|
> Two flavors from the same Model, both run on P4's but with different versions
> of Windoze. They're similar but far from identical.
>
> http://climateapps2.oucs.ox.ac.uk/cpdnboinc/workunit.php?wuid=212401
>
> (HostID #4741 is mine.)
This is an example of what I meant by:
> 1. the same workunit on different computers seems to be giving different
> weather. This is true even if exactly the same download is used as per the
> threads on the php board.
Some of the experiments on the php board showed this occuring for exactly the same model download with the weather being significantly different after a week or two.
____________
____________________________<br>
<a href="http://www.boincforum.info/boinc/">boinc forum</a> and <a href="http://www.uk4cp.co.uk/">United Kindom</a> team, my climate change <a href="http://www.livejournal.com/users/mike_atkinson/">blog</a>. |
|
|
|
|
|
I certainly agree it would be nice to have some explanation of those 6 items Mike mentioned earlier. I expect we may have to wait longer for some. DaveF has said 1 is due to different math libraries being used to solve the same equations. Presumably a different technique to estimate a solution to an equation could lead to a different least significant digit. This will lead to chaotically different results - the butterfly effect in operation.
Re 4. I believe they are quite well grouped.
Re 5. They use the last 8 years for diagnostics. An exponential extrapolation can be used to estimate the equilibrium though I think I heard at the open day a suggestion that such an extrapolation was tending to overestimate the climate sensitivity (though that may have been underestimate).
Re 6. I expect we do need the different ic sets. Maybe with some more work to ensure that results from different computers produce the same sort of differences as initial conditions, the team would then be in a position to use different computers in the same way as an ic ensemble. I doubt they could have taken the risk of using different computer to provided an ic ensemble from the start.
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
> Two flavors from the same Model, both run on P4's but with different versions
> of Windoze. They're similar but far from identical.
>
Has someone tried to crunch exactly the same model on the same computer (same hardware and same Windows version).
Of course everybody would expect the same result, but I wouldn't be surprised to see differences if the model is chaotic.
____________
Arnaud
|
|
|
|
|
|
> Has someone tried to crunch exactly the same model on the same computer (same
> hardware and same Windows version).
I've tried to crunch a model the same timesteps several times.
(To be exactly: I stopped cpdn, made a backup, started cpdn, after 144 steps the model was stopped again, saved the restart.day file, backup restored, cpdn run again for 144 steps then stopped, compare restart.day with the saved restart.day file)
The restart.day file was always different!
I don't exactly know what is saved in this file, I thought it would be the field variables of the model. If this was right, this would mean, that there must be some random numbers thrown into the calculations.
____________
|
|
|
|
|
|
Interesting Josti
Did you do a file compare and was there any indication of how much and what was different. Just a possibility that something like the date & time completed is stored in there.
I have just had a model that went back to the start on me. I copied the pe12 file and let the model continue. Doing a file compare found some differences. However using CPView I dumped the temperatures for a season to a CSV file from each. 7008 temperatures each recorded to 4 decimal places were identical.
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
Hi
Not bad final temperature agreement (at the final display resolution) between these 3 results of WU235513 - 2mo6_100143935. All from different CPU's and O/S's:
Host 70466 - P4 2GHz, Win2k (Pro) OS
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=438997&phase=AT#graph
Host 78240 - Zeon 2.8GHz, Linux 2.4.21-1SCORE_xeon OS
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=438998&phase=AT#graph
Pity this one had a server error in collection of some of the data points in Phase 3.
Host 3708 (my PC, Amy) P4 3.2GHz(H/T), WinXP (Home) O/S
http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=438999&phase=AT#graph
Once the php boards are back up, I will post further results from my 'unofficial' run looking at the behaviour of exactly the same downloaded model run on two different PC's. As I thought might happen, up to 50% of Phase 2, there has been much more deviation between the two machines in Phase 2 than there was in Phase 1.
Pete
|
|
|
|
|
|
Does anyone know when different maths libraries may be used. eg Could a different stepping of the same chip at the same speed use a different maths library? And/or Could 2 chips of the same stepping but run at different speed use a different maths library?
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
> Did you do a file compare and was there any indication of how much and what
> was different. Just a possibility that something like the date & time
> completed is stored in there.
Oh, you are right. There must be something stored like cputime or date or something.
I've run a model twice for 40 modeldays (on the same system) with the result that the temperature fields were identical.
So I conclude that there are *no* random numbers involved. Otherwise the weather in a model would be significantly different after some weeks, due to the nonlinearity of the equations.
I hope this conclusion is more correct than my previous one :-)
____________
|
|
|
|
|
|
I know from my fortran programing, that results can be different, when different compiler options are used. This difference comes from different internal precision of the calculations in the cpu, if I remember right. Usually only the last decimal place is different.
But in a weather model this would lead to different weather after a few weeks.
[speculation]
I think the cpdn model for linux is more optimized than the one for windows, which could explain the difference when the same model is run on different OS.
[/speculation]
But where do the differences within the same OS come from?
From machine errors??? Don't really know.
EDIT:
I found two models with the same parameters which got exactly the same result (one on a athlon64, the other on a athlon, both under windows):
<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=421090&phase=AT#graph">Link1</a>
<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=421092&phase=AT#graph">Link2</a>
EDIT 2:
Sorry, I've posted the wrong resultids
____________
|
|
|
|
|
|
Stumbled over the same thing a few days ago :
Same Model, but definitely notable differences :
(my) AthlonXP 3000+, Win2000 SP4 :
<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=399030&phase=AT#graph">http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=399030&phase=AT#graph</a>
(other) Pentium4 3000MHz, WinXP SP2 :
<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=399028&phase=AT#graph">http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=399028&phase=AT#graph</a>
____________
Scientific Network : 44800 MHz - 77824 MB - 1970 GB
 |
|
|
|
|
|
> Re 5. They use the last 8 years for diagnostics. An exponential extrapolation
> can be used to estimate the equilibrium though I think I heard at the open day
> a suggestion that such an extrapolation was tending to overestimate the
> climate sensitivity (though that may have been underestimate).
>
I think I must have been mistaken or at least misinterpreted the size/importance of the error. The results paper says the error is small.
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
>
> 2. the same workunit may give both stable and unstable phase 2. Probably
> caused by some weather conditions in the unstable case leading to cold
> equators, but this would be nice to be confirmed.
>
> 3. there are hints that the runs in this thread have different climate
> sensitivities for the same workunit (though this is difficult to verify as the
> second run has an unstable phase 2). Is this the case and if so what is the
> cause?
>
> 4. Are all unstable runs grouped in the parameter space, or are some scattered
> thoughout the parameter space? Is there a difference between those that are
> always unstable and those that are just sometimes unstable?
>
The supplementary information to the first results paper in nature
http://www.nature.com/nature/journal/v433/n7024/suppinfo/nature03301.html
says:
Cooling Simulations
There is a well understood mechanism for models with a mixed layer ocean to produce dramatic, unphysical cooling. The process begins in the eastern tropical Pacific. During the calibration phase there is a mean level of low cloud over the region and consequently a negative cloud radiative forcing. The heat flux convergence field is negative here, mimicking the cooling effect of upwelling colder ocean water. During a subsequent phase the regional amount of cloud may become greater than was typical in the calibration phase. As a result the amount of short wave radiation penetrating to the surface is reduced and the sea surface temperatures fall. This can lead to further low-level cloud and a positive feedback mechanism, ultimately cooling the whole planet. This runaway effect is an unphysical consequence of using a mixed layer ocean; it could happen at any point in the control or double CO2 phases. Some model versions may be more susceptible but any model versions with sufficient variability, potentially including the unperturbed model, could produce it given sufficiently long simulations.
The climateprediction.net simulations highlight this generic problem of mixed layer models, as a consequence of three factors: i) the total number of model years is far greater than in any previous experiment so the effect is simply more likely to occur, ii) some perturbed model versions may be more susceptible due to increased variability, and iii) the experimental design requires that all model versions use a 15 year calibration phase – for model versions with higher than standard variability it may be necessary to increase the length of this phase to reduce the likelihood of encountering the problem.
Of the simulations removed from the analysis due to a drift in the control, many exhibit only slow drifts in global mean temperature; a result either of an insufficiently long calibration or a spurious deduction as a consequence of natural variability in the control. The remaining ones appear to be a result of the above described effect. Its consequences are seen in the surface temperature field which show a dramatic cooling in the eastern equatorial pacific. Supplementary figure 2 shows the difference in the surface temperature field between the control and calibration phases for (a) a stable simulation and (b) an unstable simulation showing this effect. These unstable simulations are removed from the analysis by the requirement that the drift in Tg be less than 0.02 K/year in the last eight years of the control (see Methods – Data Quality). It is impractical to assess these fields by eye for all simulations in our grand ensemble but such manual verification confirms that none of the simulations which pass the quality and stability checks and have sensitivity greater than 8.5K, exhibit this problem.
Six simulations have stable controls but show a negative drift in the double CO2 phase. These are also a result of the above described effect as can be seen in supplementary figure 3; in the simulation shown the 8 year mean surface temperature in the double CO2 phase shows a strong cooling in the tropical pacific with temperatures down to 27K below the calibration phase. We consider it justified, therefore, to remove them from the current analysis. However, it will be important not to omit such parameter combinations from future experiments using models with dynamic oceans since they would not exhibit the same problem.
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
>Some model versions may be more susceptible but any model versions with >sufficient variability, potentially including the unperturbed model, could >produce it given sufficiently long simulations.
So when I have tried to find the parameters that cause cold equators, I am probably finding the parameters that cause more variability. This probably explains why there is a considerable overlap between the parameters that apparently cause cold equators and the parameters that cause hot models.
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
I think all my questions are answered by the paper, supplementary discussion and answers on this thread.
Thanks crandles and all who replied.
____________
____________________________<br>
<a href="http://www.boincforum.info/boinc/">boinc forum</a> and <a href="http://www.uk4cp.co.uk/">United Kindom</a> team, my climate change <a href="http://www.livejournal.com/users/mike_atkinson/">blog</a>. |
|
|
|
|
|
oops mistake.
____________
Visit BOINC WIKI for help
And join BOINC Synergy for all the news in one place. |
|
|
|
|
|
....<b>> However, it will be important not to omit such parameter combinations from
> future experiments using models with dynamic oceans since they would not
> exhibit the same problem.</b>
>
Very interesting, thanks Chris, I was wondering how I was going to be able to read the supplements without subscribing to 'Nature'. The final paragraph that I've highlighted above is good to know - we were saying much the same as that at the CPDN open day, and somewheres in the long 'cold equator' thread on the phpBB too but hadn't really heard any response until now. :-)
____________
<a href="http://www.nmvs.dsl.pipex.com/"><img src="http://boinc.mundayweb.com/cpdn/stats.php?userID=6&team=off&trans=off"></a>
<a href="http://www.nmvs.dsl.pipex.com/">Distributed Mania</a> |
|
|
|
|
|
Hi there
For those who were following my experiment running identical model comparisons (WU248363) on 2 different PC's, Amy and Stacey on the php boards, here are the end of Phase 2 results:
<a href="http://climateapps2.oucs.ox.ac.uk/cpdnboinc/result.php?field=Temperature&resultid=466808&phase=2#graph">Amy's official 'live' Phase 2 Result</a>
Comparative Temperature Trace:
<img src="http://cpdn.tuxie.org/Pete_B/Combined_Data/Amy_Stacey_Ph_2_Temp_Trace(m).png">
Amy Temperature Map, Summer 1840:
<img src="http://cpdn.tuxie.org/Pete_B/Amy_Data/WU248363/Amy_Summer_1840.png">
Stacey Temperature Map, Summer 1840:
<img src="http://cpdn.tuxie.org/Pete_B/Stacey_Data/WU248363/Stacey_Summer_1840.png">
Amy - Stacey Differential temperature Map, Summer 1840:
<img src="http://cpdn.tuxie.org/Pete_B/Combined_Data/Amy_Stacey_Summer_1840.png">
For most of Phase 2, Stacey calculated a slightly warmer Global temperature than Amy following a very close Phase 1. The end of Phase 2 differential map shows the regional differences between the two PC's. Both machines are now in Phase 3 but with a sobering result so far, which comes right on the back of today's Nature publication.
Stacey is running slightly ahead of Amy, she is now at 75% of the run, i.e. 4 years into Phase 3. In all my CPDN runs so far (5 Classic's & 5 BOINC's), I have never yet seen such a rapid rise in temperature as Stacey is calculating. 4 years into Phase 3, the Global average is already above 16C and is still rising at a rapid rate :-(.
Only to wait and see how Amy processes the Phase.
Pete |
|
|
 Dave FrameForum moderator
Project administrator
Project scientist
Send message
Joined: Aug 23 04
Posts: 49
Credit: 0
RAC: 0
|
|
> ....<b>> However, it will be important not to omit such parameter
> combinations from
> > future experiments using models with dynamic oceans since they would not
> > exhibit the same problem.</b>
> Very interesting, thanks Chris, I was wondering how I was going to be able to
> read the supplements without subscribing to 'Nature'. The final paragraph that
> I've highlighted above is good to know - we were saying much the same as that
> at the CPDN open day, and somewheres in the long 'cold equator' thread on the
> phpBB too but hadn't really heard any response until now. :-)
(1) We're intending to put together a "bumper issue" of the cpdn paper (or maybe an extended mix - not sure how we're supposed to phrase these things...) that will include the supplementary info as well as the paper, and maybe Myles's '99 Nature piece when cpdn was just a pipe dream. And it will have a pretty cover (from the Nature mock-up).
(2) I thought we'd already said that we wanted to explore the cold equator models in the coupled ensemble? I think we're all quite keen that we crack on with a few papers now - some really basic ones are: (a) the duplicates paper (Sylvia); (b) the quality control paper (Jamie and Stephen at RAL); observational comparisons (Claudio); the cold equator paper (Nick or me, probably) and a sort of ic paper, but that might take a bit of time till we get the big ic ensemble back - Dave S is quite keen on that one, I think. And the non-linearity paper, which a few of us are keen on. And then there is all the other research (Nick's THC stuff, Duncan & Ellie's s-cycle, some ocean spinups...) too. And some of our collaborators have some cool ideas, too.
Dave
____________
|
|
|
|
|
|
> (1) We're intending to put together a "bumper issue" of the cpdn paper (or
> maybe an extended mix - not sure how we're supposed to phrase these things...)
> that will include the supplementary info as well as the paper, and maybe
> Myles's '99 Nature piece when cpdn was just a pipe dream. And it will have a
> pretty cover (from the Nature mock-up).
I Look forward to reading it &, very probably, printing it out 'for posterity' too. ;-)
> (2) I thought we'd already said that we wanted to explore the cold equator
> models in the coupled ensemble?
My memory of the open day is getting a little hazy now - I do recall one of my quaeries being answered in that these 'cold equator models' wouldn't be arbitrarily ruled out for the coupled model but the comment in 'Nature' is more explicit than that I think.
> I think we're all quite keen that we crack on
> with a few papers now
Us users out in the community are quite keen on reading more 'results' too methinks - nice to see some properley done analysis of our computer's work. ;-)
> - some really basic ones are: (a) the duplicates paper
> (Sylvia); (b) the quality control paper (Jamie and Stephen at RAL);
> observational comparisons (Claudio); the cold equator paper (Nick or me,
> probably) and a sort of ic paper, but that might take a bit of time till we
> get the big ic ensemble back - Dave S is quite keen on that one, I think. And
> the non-linearity paper, which a few of us are keen on. And then there is all
> the other research (Nick's THC stuff, Duncan & Ellie's s-cycle, some ocean
> spinups...) too. And some of our collaborators have some cool ideas, too.
:-D
____________
<a href="http://www.nmvs.dsl.pipex.com/"><img src="http://boinc.mundayweb.com/cpdn/stats.php?userID=6&team=off&trans=off"></a>
<a href="http://www.nmvs.dsl.pipex.com/">Distributed Mania</a> |
|
|
mo.vForum moderator
Project administrator
Volunteer tester
Send message
Joined: Oct 10 04
Posts: 223
Credit: 4,664
RAC: 0
|
|
We are also hoping that the Open University will start the course sometime soon so that the less climatologically and statistically literate among us (eg me) can understand both our models and the real world and the forum discussions and the publications with greater ease.
Will the OU give discounts off fees for completed models?
____________
__________________________________________________
|
|
|
