Lucasarts 60FPS upscaler

[Brad]

Hopping in here based on a link in the framerate discussion topic:

We shouldn't worry ourselves with 'gameplay' processing speed. There's been some discussion about the gameplay still running at 30fps so there'l be more perceived lag? Gameplay calculations don't have to have any relation to the video framerate. Physics engines are often performing more calculations per second than the framerate. Forza 3 and the new F1 game for example both analyse the relationship between various bits of geometry much faster than 60 times per second.

[EvilBoris]

What the article was saying is that lag is less noticeable, because you are seeing the interpolated 'fake' frame before it has actually rendered the real one you are seeing things sooner that you would do if you haven't used the technique

[Brad]

Exactly. The movements you make with the controller might be picked up and the impact of them calculated immediately but then you're waiting for 1/30th of a second to see that impact. If they can 'tween' a new frame for you based on the previous frame, coupled with the full render of the next frame then the lag could appear to be less. Seems crazy. Would love to play it.

[Matt]

It's not possible. At that stage you're creating a whole new frame. I have yet to see a flow that explains how it's possible.

[EvilBoris]

But you aren't rendering a whole frame in the traditional way that all the frames are normally done you are rendering the new frame based upon 2 static images (imagine them as jpeg files) and the difference between them.

2 Static images being used to create 1 new frame us all very well but what if the computer gets the middle fake frame wrong and adversely affects the appearance if motion.
What if you could use this extra frame to not only show what has happened after you have these 2 images but actually create this 'fake' frame incorperaring extra information from user input?

What he is saying is that it's very tricky to do it in real time without lag for video compression or interpolation because compression technology and the TV motion interpolation works by tracking the motion of objects (or groups of pixel that constitute detail) In order to do this effectively it takes time.

If you've ever seen a badly compressed video with people you sometimes see a face or recognisable object smear across in a blocky mess of vaguely recognisable mess . It often happens with faces, I assume this happens if the compression has left the video in a state where it's only been able to track the face but nothing else in the scene. It will often happen with fast motion or between camera angle cuts of sudden changes in screen content (cutting To a black screen for example)

Now with a game because all of the motion and screen content is always being controlled 100% by the machine all of this information to where on screen objects ate going to end up and the path that they take is already pre determined and all this information is used so it can generate an accurate fake frame far quicker than if they don't have the information to where and how objects/details are moving.

Presumably this is very quick and is quicker than rendering those extra frames to give the true 60fps but due to this system allowing for user input to be visibly incorporated into the 'fake' middle frame it provides a less laggy experience than a standard 30fps title.

That's my understanding of how it works and why it works. I may of misunderstood and inadvertently invented something though

[Matt]

It's impossible to interpolate a frame without knowing the next one. And you'd only know the next frame when it's rendered. As which stage you're buffering the next real frame to get a smoother experience, but doubling the input latency.

What they're doing, as I understand it, is making a frame based on the previous frame buffer and, basically, momentum - player is moving this way, thus we'll interpolate a frame moving things a bit more this way. How it works with say a sideways movement, where you no information past the viewport bounds, I don't know. I guess if it's 1280x720 you can be running the framebuffer at 1400x800, showing the central area only, and using the rest as an interpolation buffer. Seems wasteful.

Honestly, I just don't "get" how it can be done. You simply cannot interpolate the extra frame based on the next frame. That's impossible

I guess, going out on a limb here, if you're interpolating while you start rendering the next frame, if you're calculating the most important elements first and somehow rendering them apart from the rest (ie enemies seperately from the background) you could do something there.

[EvilBoris]

When I see a frame on screen how many frames in advance has that been finished and rendered?

[Matt]

None. You draw a new frame while the previous one is displayed. Once you've drawn it, you flip the page (show the new frame) and start work on the next - actually that's not technically true, you don't necessarily flip the page immediately as you'd wait for 33ms (if it's locked at 30fps) before displaying the next frame regardless, even if it rendered at 10ms. You poll the display and flip the page when the monitor is ready (this stops tearing).

You can get into triple buffering, but then you no longer limit the framerate and chuck the image on screen as soon as it's ready - so you get tearing.

[EvilBoris]

Ah I see how it's impossible as I imagined it.

I found the quote that explains how it does it without actually having the second frame.

Dmitry Andreev: The basic idea is to build an extra frame based of the previous one [using] new information available for the current, new frame, and present it in the middle of 30FPS rendering while still working on the current frame. This way, technically, we are seeing an effective 60FPS with all the advantages of rendering at 30FPS.

So they are only actually using the first frame but using that and the motion vector information creating the next pseudo frame, which is then followed by another real frame

[Matt]

The "using new information available for the current, new frame" - does that mean the next frame? Or the existing one? If it's the next, yikes, artifact city.

Who knows. Hoping this turns out to be brilliant. Maybe someone could explain how it works? We need to know!

[Giga_Death]

Is don't know if this is good feature for next gen since, now, most devs prefer 30fps games to get maximum graphical fidelity. I mean Bayonetta is great, 360 version at least.

I definitely hope fighting games don't do this at least.