this post was submitted on 10 Dec 2024
73 points (88.4% liked)
PC Gaming
8765 readers
642 users here now
For PC gaming news and discussion. PCGamingWiki
Rules:
- Be Respectful.
- No Spam or Porn.
- No Advertising.
- No Memes.
- No Tech Support.
- No questions about buying/building computers.
- No game suggestions, friend requests, surveys, or begging.
- No Let's Plays, streams, highlight reels/montages, random videos or shorts.
- No off-topic posts/comments, within reason.
- Use the original source, no clickbait titles, no duplicates. (Submissions should be from the original source if possible, unless from paywalled or non-english sources. If the title is clickbait or lacks context you may lightly edit the title.)
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I feel this is a bit of an overstatement, otherwise you'd only render the first frame of a game level and then just use this method to extrapolate every single subsequent frame.
Realistically, the model has to return back to actually fully pipeline rendered frames from time to time to re-reference itself, otherwise you'd quickly end up with a lot of hallucination/artefacts, kind of an AI version of a shitty video codec that morphs into nonsense when its only generating partial new frames based on detected change from the previous frame.
Its not clear at all, at least to me, in the paper alone, the average frequency, or under what conditions that reference frames are reffered back to... after watching the video as well, it seems they are running 24 second, 30 FPS scenes, and functionally doubling this to 60 FPS, by referring to some number of history frames to extrapolate half of the frames in the completed videos.
So, that would be a 1:1 ratio of extrapolated frame to reference frame.
This doesn't appear to actually be working in a kind of real time, moderated tandem between real time pipeline rendering and frame extrapolation.
It seems to just be running already captured videos as input, and then rendering double FPS videos as output.
...But I could be wrong about that?
I would love it if I missed this in the paper and you could point out to me where they describe in detail how they balance the ratio of, or conditions in which a reference frame is actually referred to... all I'm seeing is basically 'we look at the history buffer.'
Thats a good point, I missed that, and it's worth mentioning they ran this on a 4070ti.
Unfortunately they don't actually list any baseline for frametimes generated through the normal rendering pipeline, would have been nice to see that as a sort of 'control' column where all the scores for the various 'visual difference/error from standard fully rendered frames' are all 0 or 100 or whatever, then we could compare some numbers of how much quality you lose for faster frames, at least on a 4070ti.
If you control for a single given GPU then sure, other than edge cases, this method will almost always result in greater FPS for a slight degredstion in quality...
...but there's almost no way this method is not proprietary, and thus your choice will be between price comparing GPUs with their differing rendering capabilities, not something like 'do i turn MSAA to 4x or 16x', available on basically any GPU.
More on that below.
Yes, this is why I said this is GPU tech, I did not figure that it needed to be stated that oh well ok yes technically you can run it locally on a CPU or NPU or APU but its only going to actually run well on something resbling a GPU.
I was aiming at practical upshot for average computer user not comprehensive breakdown for hardware/software developers and extreme enthusiasts.
To be fair, when I wrote it originally, I used 'apparently' as a qualifier, indicating lack of 100% certainty.
But uh, why did I assume this?
Because most of the names on the paper list the company they are employed by, there is no freely available source code, and just generally corporate funded research is always made proprietary unless explicitly indicated otherwise.
Much research done by Universities also ends up proprietary as well.
This paper only describes the actual method being used for frame gen in relatively broad strokes, the meat of the paper is devoted to analyzing it's comparative utility, not thoroughly discussing and outlining exact opcodes or w/e.
Sure, you could try to implement this method based off of reading this paper, but that's a far cry from 'here's our MIT liscensed alpha driver, go nuts.'
...And, now that you bring it up:
Intel filed what seem to me to be two different patent applications, almost 9 months before the paper we are discussing came out, with 2 out of 3 of the credited inventors on the patents also having their names on this paper, which are directly related to this academic publication.
This one appears to be focused on the machine learning / frame gen method, the software:
https://patents.justia.com/patent/20240311950
And this one appears to be focused on the physical design of a GPU, the hardware made to leverage the software.
https://patents.justia.com/patent/20240311951
So yeah, looks to me like Intel is certainly aiming at this being proprietary.
I suppose its technically possible they do not actually get these patents awardes to them, but I find that extremely unlikely.
EDIT: Also, lol video game journalism processional standards strike again, whoever wrote the article here could have looked this up and added this highly relevant 'Intel is pursuing a patent on this technology' information to their article in maybe a grand total of 15 to 30 extra minutes, but nah, too hard I guess.