[Climbing] 10/01/22 - 10/17/22: Red River Gorge






























 

Comments

Post a Comment

Popular posts from this blog

Benchmarks: Dask Distributed vs. Ray for Dask Workloads

Image
  Dask Workloads  In this section, I detail the dask workloads that will be used to benchmark two cluster backends - Ray and Dask Distributed. In summary, my dask workload involves processing ~3.3 Petabytes of ~6.06 million input files with dask-based Processing Chain and producing ~14.88 Terabytes of ~1.86 million output files. Input Data ~6.06 million ~550MB files, totaling ~3.3 Petabytes  Each file represents a N-dimensional numerical tensor (np.ndarray).  Processing Chain: How we go from Input data to Output data (Dask Task Graphs) We use  Xarray  / Dask APIs for  lazy , numerical computations on out-of-core multidimensional tensors. Per 3 input files, we generate 1 output file using below  Dask Task Graph  (auto generated by using Dask APIs) The Task Graph is consisted of small, Dask-level 'tasks'; tasks can be grouped into 3 actions:   Load Download file from S3 and load each file into 2D  np.ndarray  of type np.floa...
Read more

Parallel processing libraries in Python - Dask vs Ray

At work, I'm working with enabling numerical computations/transformations of N-dimensional tensors that cannot fit in a single machine's memory. They are "big data" in the order of TBs-PBs. The data types are numerical arrays; e.g. np.array. Data processing / analysis code are written in Python. I looked at two libraries to enable parallel computing within the Python ecosystem. The two are Dask and Ray. I made some comparison between them.   Dask Dask is a flexible library for parallel computing in Python  https://docs.dask.org/en/latest/ Ray Ray is a fast and simple framework for building and running distributed applications. https://ray.readthedocs.io/en/latest/ (Originally published 02/25/2020, updated 03/03/2021 as per new features in the libraries) Dimension Dask Ray Color Code Justification Efficient Data Sharing Between Nodes  Source:  https://distributed.dask.org/en/latest/serialization.html Data sharing happens via TCP messages between workers. Every ti...
Read more

2020 Climbing Season - Day Crag Photo Compilations & My Route Pyramid

Image
When the pandemic began and the gravity of it sank it on March, it was quite depressing. When outdoor recreation opened back up in June or so, I began to find solace in climbing outside as much as possible. My partners and I climbed before or after work on the weekdays and on most weekends. This post contains some photos taken on the cragging days. Omitted are the photos from multipitch routes that are in other posts. Also, many short sessions before/after work at the Exit 32/38 crags are not represented, since those sessions are spent cramming as many route burns as possible.. :)  June Fellow climber on top of P1 of Zig Zag  (5.7, Trad) @ Mt. Erie Nate on... I forgot which route, they all kind of look the same @ Vantage My stick clip, stuck on 2nd bolt while projecting  Crescendo of the Sarcophagus Bleeding  (5.11b, Sport) with Nate @ Exit 38. Don't ask how this happened.... Zoomed in photo of my stick clip July Me on Gotta Keep Dancin'  (5.9, mixed, 4 pitches)...
Read more