- 79
- 25 704
Pangeo
เข้าร่วมเมื่อ 8 พ.ค. 2019
Pangeo is a community and platform for big data analysis in the geoscience and beyond. You can learn more about Pangeo at pangeo.io. This channel hosts content generated by the Pangeo Community at our meetings, including the Pangeo Showcase seminar series.
Arkouda as an XArray backend for HPC!
Recent improvements to Arkouda (github.com/Bears-R-Us/arkouda; a numpy-like Python library for working with distributed arrays and dataframes in an HPC context) have allowed it to serve as a new backend for XArray. This talk will give a brief introduction into what makes Arkouda unique, and then cover how and why one might use it as a backend for their XArray workflows.
มุมมอง: 65
วีดีโอ
xdggs: discrete global grid systems with xarray
มุมมอง 160วันที่ผ่านมา
Pangeo Showcase talk on Dec 4, 2024 by Justis Magin. Discrete Global Grid Systems (DGGS) are innovative spatial reference systems that provide a systematic spatial subdivision of spherical or ellipsoidal objects. Unlike traditional latitude and longitude grids or unstructured meshes, the object’s surface is recursively subdivided into equally sized and uniquely identifiable cells, forming a tre...
Xarray’s GroupBy, oh my!
มุมมอง 145หลายเดือนก่อน
I will briefly demonstrate recent upgrades to Xarray’s GroupBy, including 1. Grouping by multiple arrays 2. New Grouper objects for custom grouping logic, for example specifying order of groups (like seasons) 3. Sort by group using a shuffle 4. Fast automatic choosing of parallel algorithm for reductions in flox 5. Faster reductions with numbagg
VEDA: NASA’s Platform for Open Science
มุมมอง 147หลายเดือนก่อน
VEDA is a cloud-based platform transforming Earth data visualization, exploration, and analysis. Built entirely on open-source tools, VEDA’s framework is now being reused by NASA for projects like the Greenhouse Gas Center and other Earth Science applications. Learn more here: impactunofficial.medium.com/introducing-veda-an-open-science-platform-to-accelerate-earth-science-research-and-applicat...
Awesome Panel
มุมมอง 104หลายเดือนก่อน
Pangeo Showcase talk on October 30, 2024 by Marc Madsen. In this talk Marc shows how you can make your workflows or your solutions more awesome with Panel. He demonstrates a variety of different applications from awesome-panel.org. He also shares his experiences contributing to Panel and the HoloViz ecosystem.
Icechunk: An Open-Source Transactional Storage Engine for Zarr
มุมมอง 277หลายเดือนก่อน
Over the past year or two, the mainstream cloud data community has witnessed remarkable convergence across major data platforms around so-called “table formats” such as Iceberg, Hudi, and Delta Lake. These table formats organize many individual Parquet files into a single logical table, supporting database style operations on top of vanilla cloud object storage (e.g. S3) and interoperability ac...
HYCOM-OceanTrack: From 17,518 NetCDF files to an Analysis-Ready Cloud-Optimized dataset in the cloud
มุมมอง 75หลายเดือนก่อน
In this presentation, I’ll share the tale of 17,518 NetCDF files and the epic 4-year quest that turned them into an augmented, Analysis-Ready, Cloud-Optimized dataset of simulated Eulerian and Lagrangian oceanic variables. Along the way, we navigated 3 funding streams, collaborated with a few people, and faced our fair share of data-induced headaches. Spoiler alert: the story has a happy ending...
marimo: an open-source reactive notebook for Python
มุมมอง 378หลายเดือนก่อน
This talk presents marimo, an open-source reactive notebook for Python that’s stored as pure Python, executable as a script, and shareable as a data app. marimo notebooks are reactive - run a cell and marimo can automatically run dependent cells (or mark them as stale), eliminating hidden state and rapidly increasing the pace of exploratory development. Behind the scenes, marimo maintains a dir...
Project Pythia Cookbooks: challenges and progress in breaking down barriers to Open Science
มุมมอง 652 หลายเดือนก่อน
Pangeo Showcase Talk on October 2, 2024 by Brian Rose. Project Pythia is creating on-ramps to the Pangeo community with open, interactive learning resources centered on Python in the geosciences. Our current focus is on Pythia Cookbooks: a community-owned collection of accessible, reusable, and reproducible tutorials and exemplar workflows in the cloud, and a successor to the former Pangeo Gall...
Geospatial reprojection in Python 2024 - what’s available and what’s next
มุมมอง 2182 หลายเดือนก่อน
Pangeo Showcase Talk on Sep 25, 2024 by Max Jones. This Pangeo showcase describes a work-in-progress guidebook on some of the many existing warp resampling / reprojection methods in Python, along with some memory and statistical wall-time profiling results.
Pangeo ML: Open source tools and pipelines for scalable machine learning using NASA EO data
มุมมอง 3296 หลายเดือนก่อน
Pangeo Showcase talk on June 5, 2024 by Max Jones: discourse.pangeo.io/t/pangeo-showcase-pangeo-ml-open-source-tools-and-pipelines-for-scalable-machine-learning-using-nasa-earth-observation-data/4249
CoCalc for Pangeans
มุมมอง 996 หลายเดือนก่อน
Pangeo Showcase talk on Wed, May 29, 2024 by Blaec Bejarano. CoCalc is a real-time collaborative environment for using your favorite programming languages and applications for computational science, all via your web browser. Jupyter Notebooks, Linux Terminal, LaTeX/Markdown documents, VS Code, Pluto, Rstudio, X11, Course Management System, or a Computational Whiteboard Environment/Slideshow Mod...
Zarr-Python 3 and why you should be excited!
มุมมอง 7056 หลายเดือนก่อน
Pangeo Showcase talk on May 22, 2024 by Joe Hamman. Zarr-Python 3 represents a major refactor of what has traditionally been viewed as the canonical implementation of the Zarr. In this presentation, we will explore what we learned from Zarr 2, why a major rewrite was needed, and what’s new in version 3.
VirtualiZarr: Create virtual Zarr stores using xarray syntax
มุมมอง 4177 หลายเดือนก่อน
Pangeo Showcase talk by Tom Nicholas on May 15, 2024. The Kerchunk idea solves an incredibly important problem: accessing big archival datasets via a cloud-optimized pattern, but without copying or modifying the original data in any way. This is a win-win-win for users, data engineers, and data providers. Users see fast-opening zarr-compliant stores that work performantly with libraries like xa...
The Open Radar Stack: Bringing Weather Radar Data into Pangeo
มุมมอง 2578 หลายเดือนก่อน
Pangeo Showcase talk from April 3, 2024 by Max Grover. Weather radar data is a complex, yet critical dataset to understanding our weather and climate system. These observational datasets have traditionally been plagued by a plethora of non-standard, difficult to read data formats, requiring the development of open-source toolkits to enable open science in this space. The Python ARM Radar Toolki...
Cloud Native Data Loaders for Machine Learning Using Zarr and Xarray
มุมมอง 4518 หลายเดือนก่อน
Cloud Native Data Loaders for Machine Learning Using Zarr and Xarray
Optimizations for Kerchunk aggregation and Zarr I/O at scale for Machine Learning
มุมมอง 2199 หลายเดือนก่อน
Optimizations for Kerchunk aggregation and Zarr I/O at scale for Machine Learning
VAPOR Python: A 3D Data Visualization Python API for Earth Science Datasets
มุมมอง 3509 หลายเดือนก่อน
VAPOR Python: A 3D Data Visualization Python API for Earth Science Datasets
Virtual Analysis Ready Data for CMIP6 and ESGF
มุมมอง 2049 หลายเดือนก่อน
Virtual Analysis Ready Data for CMIP6 and ESGF
Advancing machine learning education with open software and data
มุมมอง 1649 หลายเดือนก่อน
Advancing machine learning education with open software and data
Delivering OGC Processing API, OGC EDR, and OPeNDAP services with pyGeoAPI and Catalog-to-Xpublish
มุมมอง 159ปีที่แล้ว
Delivering OGC Processing API, OGC EDR, and OPeNDAP services with pyGeoAPI and Catalog-to-Xpublish
How to transform thousands of CMIP6 datasets to Zarr with Pangeo Forge --hopefully never again!
มุมมอง 236ปีที่แล้ว
How to transform thousands of CMIP6 datasets to Zarr with Pangeo Forge hopefully never again!
Accelerating Science with Open Source - An Introduction to Open-Source Science (OSSci)
มุมมอง 191ปีที่แล้ว
Accelerating Science with Open Source - An Introduction to Open-Source Science (OSSci)
Cubed: Bounded-Memory Serverless Array Processing in Xarray
มุมมอง 189ปีที่แล้ว
Cubed: Bounded-Memory Serverless Array Processing in Xarray
Compression of Geospatial Data with Varying Information Density
มุมมอง 122ปีที่แล้ว
Compression of Geospatial Data with Varying Information Density
The Hydro-Terrestrial Earth System Testbed (HyTEST)
มุมมอง 62ปีที่แล้ว
The Hydro-Terrestrial Earth System Testbed (HyTEST)
Improving Access to NOAA NOS Model Data with Kerchunk and Xpublish
มุมมอง 180ปีที่แล้ว
Improving Access to NOAA NOS Model Data with Kerchunk and Xpublish
Super cool - thanks for sharing!
Great work!!
Outstanding! Can't wait to use it in my next project
Great talk!
Its actually a peer to peer community network of same minded research connections
I am not satisfied with even usingnfull potential of chat gpt as research assistant as a user from market , would my startup is in climate tech , karmic winsds so please make a easy to use gui
Yoo from india , nice initiative, startup names karmic winds , will try my best to help you out from fredback atleast more better faster science ❤
Great initiative! Scientific research should be open source thats how progress can be made. General problems associated with open source publishing is that the publisher takes a huge amount to make the research open source! Hence there's a lack of availability of quality open source research papers. SciHub, although seen as piracy platform for research content is quite helpful in the scholar community. I hope this initiative brings out quality research without charging the authors a hefty amount.
*Promo sm* 😩
Great presentation
Love it.
Absolutely a step in the right direction. I would love to extend our Intake instance i such new ways. Thank you for your work Mr. Durant.
Amazing work! Congratulations!
Amazing ! Would love to have access to the repo to experiment!
Thanks for the great tutorial! Such a powerful ecosystem. Great for quick prototyping and exploration!
Amazing work!
Hello! Amazing presentation Is geopandas considered?
can we use a dataframe that has columns like p ,t,r msl and sst and pass these values in the PI function
Great work <3
Fascinating stuff! I want to try it out. are there good public datasets to try it out in a local machine? Ore some resources to use it on the cloud?
Hey! Working on something very similar. Who gave this talk I'd love to connect.
I’m excited to hear about Geoparket
Would have been real nice to see the 'clicking around on websites' for a video with "tutorial" in the title :/
This is so much a misleading video. They do not tell you all you have to do to set this up. It is very difficult and I would say almost impossible.