Pangeo example

[mrocklin]

@ncclementi any thoughts on a good Pangeo-y example we have lying around?

[ncclementi]

The one on the main page of our geospatial use cases looks pretty and simple: https://coiled-new-clean.webflow.io/use-cases/geospatial

In that page we also have linked

• The coiled + +xarray+ stackstac, https://stackstac.readthedocs.io/en/latest/examples/gif.html
• pangeo +dask, needs to be adapted from gateway to coiled https://gallery.pangeo.io/repos/pangeo-gallery/physical-oceanography/01_sea-surface-height.html
• geopandas +dask needs to be adapted from gateway to coiled. https://planetarycomputer.microsoft.com/dataset/fia#Example-Notebook

[mrocklin]

@jrbourbeau I'm curious, what are the odds you think you'll find something solid by end of week?

[jrbourbeau]

There's the issue over in coiled/benchmarks where I'm soliciting representative Pangeo workloads. At the moment I'm waiting for a specific suggestion from the Pangeo folks involved in that issue.

I also just asked if https://gallery.pangeo.io/repos/pangeo-gallery/physical-oceanography/01_sea-surface-height.html or https://gallery.pangeo.io/repos/pangeo-gallery/cmip6/global_mean_surface_temp.html seemed representative to them. I think porting one of those examples over to Coiled would be straightforward.

[mrocklin]

Yeah, sea surface altitude is my usual go-to. A challenge though is that that data is on GCP. Maybe it's easy to lift and shift that data over to AWS though.

We could also use multiple accounts to get multi-cloud. I'd like to avoid the complication when doing demos though. Who knows though, maybe that's a good selling point.

[jrbourbeau]

A challenge though is that that data is on GCP. Maybe it's easy to lift and shift that data over to AWS though.

I dug a little bit a found that pangeo folks have already lifted and shifted for the cmip6 dataset:

import intake
col = intake.open_esm_datastore("https://cmip6-pds.s3-us-west-2.amazonaws.com/pangeo-cmip6.json") 

We could also use multiple accounts to get multi-cloud

I was chatting with Nat about this offline earlier today. This is doable, but maybe not actually needed at the moment given the above comment ^

[mrocklin]

Ooh, nice find!

…

On Wed, Apr 12, 2023, 5:02 PM James Bourbeau ***@***.***> wrote: A challenge though is that that data is on GCP. Maybe it's easy to lift and shift that data over to AWS though. I dug a little bit a found that pangeo folks have already lifted and shifted for the cmip6 dataset: import intakecol = intake.open_esm_datastore("https://cmip6-pds.s3-us-west-2.amazonaws.com/pangeo-cmip6.json") We could also use multiple accounts to get multi-cloud I was chatting with Nat about this offline earlier today. This is doable, but maybe not actually needed at the moment given the above comment ^ — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTFZFATMXXRE7AFWX4DXA4RAHANCNFSM6AAAAAAWWWDNRY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[jrbourbeau]

Given cmip6 is on AWS, I think it should be straightforward to port over https://gallery.pangeo.io/repos/pangeo-gallery/cmip6/global_mean_surface_temp.html. I'll have some time to focus on that tomorrow unless you beat me to it @mrocklin

[mrocklin]

Playing with this now @jrbourbeau unless you're already on it

[mrocklin]

I've got global surface mean temperature, but it's not a very good demo. It's a lot of fussing about with a single nice graph at the end. I may look a little deeper, but I'm not super excited yet.