You should definitely re-encode it in post with higher compression settings that take much longer than you could encode “live” to get a small file with the same quality as your original high bitrate recording. (I suggest the AV1 codec for that)
I like this suggestion also, because 256TB is a lot of TB.
That's a good idea. I have a question tho, does AV1 lose some fidelity, or is it simply just a "better" encoding?
All video codecs are lossy, meaning you will lose some quality. AV1 and H.265 are modern video codecs with the best quality to bitrate ratios, meaning you can get better quality for the same bitrate, or the same quality with a lower bitrate. The downside the these codecs is that they are very complex and computational expensive to do in software. You'll want to make sure your GPU supports hardware encoding for the codecs you intend to record with. The reason most people will recommend AV1 over H.265 is that AV1 is royalty free. With H.265, companies have to pay a royalty to use H.265. Because of this, most companies (Netflix, YouTube, Facebook, Twitch, etc.) want to use AV1 going forward, meaning in the near future, it will probably be the dominant codec.
well I checked. My GPU does support AV1, but only decode, and OBS doesn't support h265. So is my only (good) option is to use a SVT-AV1? How slow it that really?
if you’re planning on editing it, you can record in a very high bitrate and re-encode after the fact… yes, re-encoding looses some quality, however you’re likely to end up with a far better video if you record and 2x the h264 bitrate and then re-encode to your final h265 (or av1) bitrate than if you just record straight to h264 at your final bitrate
another note on this: lots of streaming stuff will say to use CBR (constant bitrate), which is true for streaming, however i think probably for re-encode VBR (variable bitrate) with multi-pass encode will give a good trade-off - CBR for live because the encoding software can’t predict what’s coming up, but when you have a known video it can change bitrate up and down because it knows when it’ll need higher bitrate
how does doing a 2pass work? I'm going to be honest, I just ignored that feature. Is it worth the extra hassle? And how much longer does it take? Also do I need to run ffmpeg twice for that? Or can I just put in an argument and it'll scan twice.
After typing all that out I realized that all this is probably easier to google/try than to ask you about it. Thanks for the info tho. I managed to get a video format where 15 minutes are 1GiB and watching it back I didn't notice any artifacts or burry parts in 1440p 60fps.
2 pass will encode a file once, and store a log of information about what it did… then on the 2nd pass it’ll use that information to better know when it should use more or less bitrate/keyframes - honestly i’m not too sure of the specifics here
now, it’s most often used to keep a file to a particular file size rather than increasing quality per se, but id say keeping a file to a particular size means you’re using the space that you have effectively
looks like with ffmpeg you do need to run it twice - there’s a log option
i mostly export from davinci resolve so i’m not too well versed in ffmpeg flags etc
doing a little more reading it seems the consensus is that spending more time on encoding (ie a higher preset) will likely give a better outcome than 2 pass unless you REALLY care about file size (like the file MUST be less than, but as close to 100mb)
I did a transcode with -pass 1 added, and it generated an empty txt file.
and OBS doesn't support h265
are you sure about that? I'm pretty sure it should. maybe you'll find it as _x_265, as that's the name of the encoder it uses
I can't find it and a quick google told me that neither do others
OBS allows you to use everything FFmpeg supports with the "Custom Output (FFmpeg)" recording type.
I was and am searching there I just can't find it in the dropdown. I might be blind, but I checked multiple times.
Settings -> Output:
I've gotten that far, but no farther
turns out OBS does not have a software encoder for it, only encoding with a hardware encoder is supported. it is mentioned in the 2nd table of the video formats heading herev https://obsproject.com/kb/audio-video-formats-guide
Because it's just for personal archival, I would recommend recording in super high bitrate 264 or 265 depending on what your card can do in real time, then compressing that file later using either av1 or 265, depending on which works better for the content.
If you're playing in 1440p165, then you should record that to start although if it's an option I would play and record at 1080p120. 1440 is a bit of an odd duck resolution that some stuff doesn't really like, but it's getting better all the time so it's not a huge deal. More important is 120fps because if you decide to go down to 60 or even 30fps they split evenly, which is important since it means the extra frames just get thrown out. There's no 82.5 frame so it either has to blend two together or pick one that's at the wrong time, and neither looks good.
Record in OBS using like, 25-40mbps in whatever your graphics card can do realtime, with all the audio tracks in something lossless like FLAC. High enough it might as well be uncompressed.
Once you have your gameplay recording from OBS, use either Handbrake or FFMPEG to convert it to your long term storage format. Since this is such a big project I would make some samples using cheats to get an idea what the bullet-hell-iest parts will look like, then try a few different handbrake or ffmpeg settings, and see what gives you the tradeoff of file size and quality that you like. It'll also give you a ballpark idea about how much long term storage you'll need.
Non-realtime encoding like ffmpeg and Handbrake is much more efficient than realtime done by your graphics card, like on the order of like half the file size for the same quality - that's why you want the two step process. It also allows you to play in 120fps for that responsiveness, but watch at 60 or even 30fps to save some file size.
When you set up handbrake, you'll have a few settings to play with and make samples from - encoder, quality level, speed, fps, and rescaling.
Encoder is probably the most important. Use either x265 or SVT-AV1. In my experience, they're close in terms of efficiency, but AV1 pulls ahead in certain situations. AV1 is more efficient about large static sections and when it breaks up it just looks blurry. 265 is better at retaining texture, but when it fails its gross digital blocks. 265 is faster than AV1, but when you push AV1 and it takes 1000 years it does a better job. Basically, run a lot of tests and then decide.
Scaling is where you go from 1440 to 1080 and is probably the least important for file size. I honestly wouldn't bother with it, but you can try. Like I said, technically some TVs don't like 1440, and everything supports 1080, but I wouldn't worry about it too much.
FPS I have the least experience with. My guess is that going do something lower is going to save you some size, but its really going to depend on the codec and the content.
Quality level is the most important setting - just set it to what you think is watchable. It doesn't use a bitrate, it adjusts the bitrate to get a constant quality level, so it's much more efficient. It's important to note that they aren't exactly the same at every speed setting, so be careful.
Lastly is the speed. 265 goes from very fast to very slow, and AV1 goes from 9 (fast) to 0 (stupid slow). Personally I use slow or very slow for 265 and about 4 for AV1, but if you have a lot of video to crunch through you might want to crank that up a bit.
For audio use OPUS. 96 is the default "bitrate" and I find it to be enough. Some players don't like it but if you're thinking about using AV1, anything that would support AV1 supports opus.
The main thing will just be doing a bunch of video encode samples and finding the settings and workflow that's right for you.
I am doing that since I made that post, making educated guesses at AV1 configs. Every single one of my AV1 encoded files turned out larger than the original. I recorded the original with 15mbps, and that seems enough (as I did the cheating my self a teaser of lategame thing).
I set the OBS's recording to 60fps, hopefully that way I dodge the weird half frames, that you are talking about.
I set the quality RF to 30 in HandBreak, and as you can see it almost doubled the filesize. how?
Can you post a screenshot of your Handbrake settings? You should definitely be getting better efficiency than that.
"15k hard" is the control h264 file
rest doesn't matter I think
but in the meantime I did some encoding to h265, and I managed to get a smaller file size however the quality sucked, so I lowered the crf and it's larger again. 
In the third screenshot, preset is on 7, change that to 4. That's the speed setting - 7 is trying to do it in close to real time, 4 will take much longer but be much more efficient.
it became 504 MiB, the original was 306.
while it was doing that I tried compressing a 50Mbps clip. And I think you're gut feeling will be right, I managed to compress it with AV1 rf47 it got reduced to a 4th of it's size and it looks almost the same as the 15Mbps one. However, sadly I saw how good the raw 50Mbps one looks, so I don't think I'll go that low. Finding the sweet spot will take forever! Also isn't 50Mbps too high, the video looks amazing, but isn't that abnormally high?
If you're seeing any artifacts in the original video, you probably need to re-record in a higher bitrate. It needs to look identical to uncompressed. Your later encodes will be trying to encode all the artifacts in the original video, which could be why the file sizes keep getting bigger - you're giving it noisier video than the original.
50mbps for recording as an intermediate like that is well within the realm of normal. You can try having obs record in 264 with a quality setting instead of a bitrate setting, which can save space when things are more static - something like cq 6 or lower can do pretty well.
Unfortunately, yeah finding the sweet spot does take forever. One thing I would recommend is once you have an idea where you want to land, try a few much longer videos and see what the differences are. Slower paced sections might compress much better than the fast action stuff in one codec or another. Again it's all kind of a balancing act on where you want to be.
Oh that's a great tip, I set it to "indistinguishable" and it 3x-ed the (almost) same gameplay clip, but it does look the best so far. But if what you are saying is true, with the encoding the previous encoder's artifacts, doesn't that mean I should record in lossless?
I did try a video of my desktop doing almost nothing, and AV1 rf23 compressed it 17x. That's nice. I'll now try re-coding this "indistinguishable" h264 preset video and to record a lossless one and to re-code that as well.
From a technical level you should use something lossless in this situation, but it really quickly becomes impractical. Actually lossless 1080p60 is going to be something like 500mbps, so if you're playing for an hour I hope you have a spare 2tb drive laying around. The artifacts in really high bitrate compressed video are so minimal that they basically don't matter. Often codecs do noise removal first thing so whatever minor artifacts still exist will get smeared over by that anyway.
Also when you are testing make sure there's some movement in the video. AV1 especially has modes for presentations and things that basically make a PowerPoint, so sizes might be unrealistic if you're just recording your desktop. I don't think that gets enabled in handbrake but it's been a while since I looked.
I settled for this: ffmpeg -i "$file" -map 0:0 -c:v libsvtav1 -preset 3 -crf 23 -maxrate 85M -map 0:a -c:a copy "${file%.mp4}-av1.mp4" -y The files is still very large, but you can't notice any artifacts or any blurry parts, I transcode it with 7fps.
I stream Splatoon 3 for 2 hours every day and I record higher quality VODs alongside. I keep a lot of the VODs in my storage.
I record at 1080p 60fps 9000kbps with H265. 2 hours of that takes up 8.8GB, for simplicity we will say it's 9GB.
The 9000kbps is enough for a bitrate-heavy game like Splatoon 3, so I'd say 12000kbps is enough for you.
We can scale it up to your settings by (1440/1080)^2 * (165/60) * (12000/9000) = 6.52 (worst case, but H265 should reduce that a little bit). The scale factor mainly comes from the increased FPS and bitrate.
I'm currently looking at storing a year of footage in a 4TB HDD (9GB*365=3.3TB), so as an estimation, you need 7 of those.
There are better codecs though, such as AV1, but my GPU doesn't support AV1 hardware encoding and software encoding would cause too much lag, so I didn't use it.
Thanks for the numbers, I'll try some test clips to see what bitrate is good for me. And I'll use your formula for estimates :)
I did a test recording of 2 minutes (1440p 60fps 15000kbps) it's 320MBs, encoded it with AV1 and the file became 392MBs, it also lost all but the first audio track.
Anyways, I estimate a 6.8TB for 1k hours. Seems doable.
I’d err on the larger side for a disk. Assuming 1000 hours at 60fps, 1080p, serviceable bitrates I’d get an 16-18TB disk to start.
As to watchable that depends on how good your eyes are and the exact content.
I really hope you plan on naming these files descriptively or having a document that has tags and lists info for videos for easy searching as future you may not want to sift through hundreds of hours to find something.
Haha, yeah. I plan on editing them down as well when I'll have the time.
AV1 also supports variable frame rate. you may be able to use settings that reduce frames when not much is happening. I don't know how much it'll apply to Terraria though.
I tried encoding to AV1 with OBS and it stopped working. I tried converting h264 to AV1 and the filesize became larger. I think I'll stick to h264 lol
You likely need to tweak the CRF/other parameters. Take a look at https://trac.ffmpeg.org/wiki/Encode/AV1
(Note that I don't know how exactly to tune the parameters to get the best quality/size at the expense of encode speed which is what I would do here.)
forgot the most important part. I'm storing twitch streams from a variety channel with lots of Minecraft.
they roughly do 7 streams a week, 2-4 hours each, and the size of the collection that has all streams from 2020 October is almost 11 TB.
A quick calculation for h.264 at 1080p at 165fps, for 1000 hours would be around 256TB.
Not a great tool IMO. Doesn't list bitrate, h. 265, or AV1
It's not amazing but ok for what I could find quickly on my phone, for a ballpark estimation.
twitch - the gameplay streaming platform - limits 1080p streams to 8000 kbps at most, at 60 FPS. I think it exclusively uses H264 encoding. For most games this is plenty.
There are some where it can be felt that it's not enough, but in those cases it's always the bitrate.
these games include
unless you are playing a quick action first person shooter, 165 fps is totally unnecessary, 60 is plenty.
1440p, I'm not sure. if thats your screen resolution, maybe it's better to not lose quality to downscaling, more so because it can't be done by just averaging every 2 pixels, it would bea weird ratio I think
for encoding.. what hardware you have?
x265 is more efficient than x264, if you can afford the performance, but if you have a graphics card with hardware accelerated AV1 encoding, that may be even better. do some test recordings though.
my AMD 6000 series GPU only supports decoding for AV1. And yeah I quietly decided that 60fps will be perfect.
Not gameplay but phone camera footage, but I always convert to h264 using a CRF of 18. CRF is like a quality factor that adjusts bitrate based on how much movement there is. Lower CRF means more quality. More modern codecs like h265 and AV1 can produce even smaller files than h264, but ofc it's best to just do lots of tests.
A loosely moderated place to ask open-ended questions
If your post meets the following criteria, it's welcome here!
Looking for support?
Looking for a community?
~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~