Home
Creators
Search Recent Missing Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register Favorites Keys Review DMs Logout
Community
Telegram ThePornDude
Home Creators Posts Import Register Favorites Logout

Visualizations, reimagined (Patreon)

Published:
2023-12-09 14:07:54
Imported:
2026-06

Content

So... I started generating all those images, and then I did some napkin math

The file storage it would take to have them all pre-rendered at full size is more than my server can handle. Like, a LOT more. Terabytes more.

I'm thinking of putting them in the cloud, perhaps in an S3 bucket. But that has an extra cost, not just for the persistence of storage, but also in the time it takes for a script (like the one I use to generate videos, PDFs, web assets) to fetch the image from that cloud store.

I'll keep thinking about an optimal way to have these all permanently available, because though the computation is light, the scale is daunting.

Even in 2023 with practically unlimited file storage, it is still a limitation!

I realize these latest posts have been more about computation and technical things than music theory. I promise to turn my focus back toward the book once some of these logistical challenges are met. 

Comments

Haisam

I have lots of experience with S3, software engineering and RESTful APIs. Let me know if you need some help. Perhaps the assets can be generated at will, and cached for possible reuse and then purged after a certain time interval in order to keep costs down, or they're placed in long-term glacier storage to keep costs down. Additionally you can have server-less AWS Lambda functions that can do the processing for you.

The Exciting Universe Of Music Theory

following up on this, 7 months later! Storing everything rendered as raster images was too much. I don't need to have these files stored in a variety of raster resolutions, when I can fetch and rasterize them in any size I need, on demand! So this project slimmed down a lot, by only keeping vectors. And because the file sizes are modest, I'm able to keep everything stored on my web server. No need for cloud buckets or glaciers. This system ensures that at a glance I can see if any of the assets needed for any of my projects are available. It's already been very useful for the recent upgrades to the scale finder and scale detail pages.