This is a most excellent place for technology news and articles.
I am so tired of people, especially people who pretend to be computer experts online, completely failing to understand what Moore's Law is.
Moore's Law != "Technology improves over time"
It's an observation that semiconductor transistor density roughly doubles every ~2 years. That's it. It doesn't apply to anything else.
And also for the record, Moore's Law has been dead for a long time now. Getting large transistor density improvements is hard.
I'm gonna go on "no stupid question" and ask why my old hard drives aren't doubling in size.
You need to properly feed, water and fertilize them. If you don't do this, your old hard drives will just waste away until they're just a few megabytes, not flourish into giant petabyte trees.
Did you try evolving them?
You have to walk around in the right environment otherwise they're all going to turn into generic eevees, and you don't want that
Also the improvements in computer speed from Moore’s law were from Denard Scaling, which says with transistors 2x smaller, you can run things 2x faster but produce 2x as much heat.
Heat dissipation has been the bottleneck for a long time now.
Moore's law is about circuit density, not about storage, so the premise is invalidated in the first place.
There is research being done into 5D storage crystals, where a disc can theoretically hold up to 360TB of data, but don't hold your breath about them being available soon.
That would certainly benefit my Plex server setup
I always thought the holographic 3D discs were going to be a really cool medium in the infacy days of bluray and hd-dvd. I can't believe that's is been over a decade since the company behind it went bankrupt.
Probably a stupid question, but how can the crystals be 5d if oir universe is (at a meaningful scale) 4d?
Not a stupid question at all. Here's the Wikipedia article for it. The significant part is this:
The 5-dimensional discs [have] tiny patterns printed on 3 layers within the discs. Depending on the angle they are viewed from, these patterns can look completely different. This may sound like science fiction, but it's basically a really fancy optical illusion. In this case, the 5 dimensions inside of the discs are the size and orientation in relation to the 3-dimensional position of the nanostructures. The concept of being 5-dimensional means that one disc has several different images depending on the angle that one views it from, and the magnification of the microscope used to view it. Basically, each disc has multiple layers of micro and macro level images.
That’s fucking dope.
Wavelength could add a dimension. For example, if you have an optical disc (2D) that can be read and written separately by red and blue lasers, that makes it 3D.
That's neat, so it's using a trick of mathematics and physics to store info in greater dimensionality than just what the physical universe is limited to? Kinda like how we can use coordinates to represent 4d points on a graph even if we can't really visualize it?
Yes. Generally, "three dimensions" refers to three spatial dimensions: left/right, up/down, forward/backward. And then the fourth dimension is usually time. But if you're not talking about movement in space/time, you can have as many dimensions as you want. For example, in a video game, you can have movement in three dimensions, but you could also allow the player to move through time (fourth dimension), change characters that interact with the world differently (fifth), and so on.
Hard drive density has stagnated. There haven't been any major technology breakthroughs since 750GB PMR drives came out in 2006. Most of the capacity improvements since then have come from minor materials improvements and stacking increasing amounts of platters per drive, which has reached its limit. The best drives we have, 24tb, have 10 platters, when drives in the 2000's only had 1-4 platters.
Meanwhile, semiconductors have been releasing new manufacturing processes every few years and haven't stopped.
Moore's Law somewhat held for hard drives up until 2010, but since then it has only been growing at a quarter of the rate.
Right now there are only 24TB HDDs, with 28TB enterprise options available with SMR. The big breakthrough maybe coming next year is HAMR, which would allow for 30tb drives. Meanwhile, 60TB 2.5"/e3.s SSDs are now pretty common in the enterprise space, with some niche 100TB ssds also available in that form factor.
I think if HAMR doesn't catch on fast enough, SSDs will start to outcompete HDDs on price per terabyte. We will likely see 16TB M.2 Ssds very soon. Street prices for m.2 drives are currently $45/TB compared to $14/TB for HDDs. Only a 3:1 advantage, or less than 4 years in Moore's Law terms.
Many enterprise customers have already switched over to SSDs after considering speed, density, and power, so if HDDs don't keep up on price, there won't be any reason to choose them over SSDs.
sources: https://youtu.be/3l2lCsWr39A https://www.tomshardware.com/pc-components/hdds/seagates-mozaic-3-hamr-platform-targets-30tb-hdds-and-beyond
I've only looked at the consumer space and all I've noticed is that SSD prices were finally going down after stagnating for years, but then the manufacturers said that prices are too low and they intentionally slowed down production to increase prices, so prices are actually higher than they were a year ago.
Sometes the prices go up, but they steadily go down over time.
This chart is really good for seeing storage prices
Right, over the long term prices go down, but it still greatly annoys me that they jacked up prices in the short term. Thankfully I have no need to purchase any storage and won't for years.
Do you have to archive all the porn in the Internet?
"We do these things not because they are easy. But because we are hard!" -JFK
Did I just pave way to the greatest joke today.
I gave the subject a check. From Tom's Hardware, industry predictions are like:
| Year | Capacity (in TB) |
|---|---|
| 2022 | 1~22 |
| 2025 | 2~40 |
| 2028 | 6~60 |
| 2031 | 7~75 |
| 2034 | 8~90 |
| 2037 | 10~100 |
Or, doubling roughly each 4y. Based on that the state of art disks would 500TB roughly in 2040. Make it ~2050 for affordable external storage.
However note that this is extrapolation over a future estimation, and estimation itself is also an extrapolation over past trends. Might as well guess what I'm going to have for lunch exactly one year for now, it'll be as accurate as that.
To complicate things further currently you have competition between two main techs, spinning disks vs. solid state. SSD might be evolving on a different pace, and as your typical SSD has less capacity it might even push the average for customers back a bit (as they swap HDDs with SSDs with slightly lower capacity).
And I was impressed by Seagate launching their Mozaic 3+ 32TB HDDs...
That's honestly intense. I would be terrified of having that much data in one place
While not hard drives, at $dayjob we bought a new server out with 16 x 64TB nvme drives. We don't even need the speed of nvme for this machines roll. It was the density that was most appealing.
It feels crazy having a petabytes of storage (albeit with some lost to raid redundancy). Is this what it was like working in tech up till the mid 00s with significant jumps just turning up?
This is exactly what it was like, except you didn't need it as much.
Storage used to cover how much a person needed and maybe 2-8x more, then datasets shot upwards with audio/mp3, then video, then again with Ai.
Well hell, it's not like it's your money.
a petabye of ssds is probably cheaper than a petabye of hdds when you account for rack costs, electricity costs, and maintenance.
Not a problem I've ever faced before, admittedly
I guess you're expected to set those up in a RAID 5 or 6 (or similar) setup to have redundancy in case of failure.
Rebuilding after a failure would be a few days of squeaky bum time though.
Absolutely not. At those densities, the write speed isn't high enough to trust to RAID 5 or 6, particularly on a new system with drives from the same manufacturing batch (which may fail around the same time). You'd be looking at a RAID 10 or even a variant with more than two drives per mirror. Regardless of RAID level, at least a couple should be reserved as hot spares as well.
EDIT: RAID 10 doesn't necessarily rebuild any faster than RAID 5/6, but the write speed is relevant because it determines the total time to rebuild. That determines the likelihood that another drive in the array fails (more likely during a rebuild due to added drive stress). with RAID 10, it's less likely the drive will be in the same span. Regardless, it's always worth restating that RAID is no substitute for your 3-2-1 backups.
Yeah I have 6 14tb drives in raid 10, I'll get 2 more if i need it.
At raid6, rebuilds are 4.2 roentgens, not great but they're not horrible. Keep old backups.but the data isn't irreplaceable.
Raid5 is suicide if you care about your data.
I'm more shocked how little I need extra space!
I'm rocking an ancient 1TB for backups. And my main is a measly 512GB SSD.
But I don't store movies anymore, because we always find what we want to see online, and I don't store games I don't actively use, because they are in my GOG or Steam libraries.
With 1 gigabit per second internet, it only takes a few minutes to download anyways.
Come to think of it, my phone has almost as much space for use, with the 512GB internal storage. 😋
Maybe I'm a fringe case IDK. But it's a long time since storage ceased to be a problem.
I download both windows and linux offline installers when I buy games at gog.com, it's one of the reasons I buy there.
I can understand that having your own copy is nice, especially if the service is closed for some reason.
I just don't bother doing that anymore, I prefer browsing my library on GOG instead of a file-manager.
We can argue as much as we want about whether moore's law covers technological development in general or be pedantic like good old fundamental Christians and only read what the words say.
The bigger problem is that we have reached the era of what we could tentatively call "wal s'eroom". Thanks to enshittification (another one of those slippery words!) I predict that technological progress reverses from now on by 50% every 2 years.
2028: ~363TB 2029: ~439TB 2030: ~531TB
This is what I came up with.
Source?
Data from various searches,
https://www.oceanclub.org/h5_en/post/info/id/1545 https://www.thestack.technology/the-evolution-of-storage-alex-mcmullan-pure-storage/ https://blocksandfiles.com/2024/08/16/the-128tb-ssd/ https://www.trendforce.com/presscenter/news/20240913-12303.html
I guesstimated a ~20% growth rate
This was just mental guess work. I'm not claiming I know.
Just attempting to format those in a more readable way.
