Lucasarts 60FPS upscaler

[Profit]

Did anyone read the the Eurogamer article on the new tech by lucasarts which effectively doubles (or even quadruples) the framerate of a game with minimal system resources?

http://www.eurogamer.net/articles/digitalfoundry-force-unleashed-60fps-tech-article

It's effectively doing in software what many TVs do; but the benefit is there is no (or not as much) lag as if the TV does it.

I personally prefer to play most games with the TV frame doubler enabled and the lag is not an issue for most action games.

[sj33]

I've got a fair few Blu-Ray discs that are encoded as 1080i/30fps (Japanese concerts mainly), and one of the deinterlacing methods of my software player on the PC is a line doubler that does a very similar thing. It is apparently the only way to deinterlace video whilst retaining all data, and results in the 30fps footage running at 60fps. And it certainly does look like native 60fps, even though it's obviously not.

Assuming this is something very similar, I can vouch for the end results looking like true 60fps, it's incredible stuff.

[Matt]

Interpolating images to make an extra frame or 3? Not sure, could cause artifacts and it won't be the same as real 60fps - the control input will be the same as 30fps.

I'll read the article

[sj33]

Control input lag would surely be an issue, and I've no idea how it would work with something like games where new frames are generated on the fly.

It works extremely well with video though, it really does look like true 60. One of PowerDVD's deinterlacing modes does this, I believe.

[Matt]

Just read the article. I think the result will be more perceived input latency, as you're "seeing" an extra frame, but the controls won't be sampled until the next fulll frame. So in effect you'll have the same standard 133ms input latency (average for a 30fps title) but I think it'll "feel" more.

At least that's how I'm reading it. At first I thought they'd be drawing 2 frames ahead and interpolating a third in the middle, in which case input latency would be double. Thankfully they're not doing that, they're working it out based on velocity of objects as you would for motion blur. I do think it'll artifact to **** at times though....

[Tig]

I read that article and was blown away. So ingenious.

It doesn't create lag, it actually reduces lag. The intermediate frame is displayed before the most recent frame is even rendered- so you're actually getting feedback sooner than you would without the upscaler. And it costs less to do than motion blur.


Edit: just read Matt's post. Yeah, I'm not sure at what stage control input is sampled. Perhaps this will just create the illusion of reduced lag.



In any case, I think the biggest news is what it could mean for 3D gaming. 3D has always got a massive caveat in the form of doubled rendering requirements. but this technique could mean we get 3D games with decent framerates, and no compromise on graphics.

[Matt]

Tig, I can't see how it really reduces input latency. It's simply extrapolating information from motion and doing a quick render from that. If anything, it may give a weird effect when you turn a different direction but you'll feel like you're still drifting in the opposite direction for the interpolated frame.

They'd be no extra control sampling on this "fake" frame, and even if there were, they'd be no AI, physics, etc applied to it, so in effect the latency will be the same as it always was.

[Tig]

well, the bloke in the interview appears to claim that latency is reduced.

In terms of latency there is something interesting going on. I said that there is no extra latency, which is true. But if you think about it, latency is actually reduced because we get the new visual result 16.6 ms earlier. You see the result of your actions earlier.

The way I understood it, control sampling and physics calculations are done before the "fake" frame is generated. And that information has been used to create the up-to-date motion vectors which are used to generate the fake frame.

That might be BS though, and he might well be exaggerating.

[Matt]

But then you'd be doing double the AI and physics calculations, which can't be the case.

Let's say usually we render Frame X and Frame Z. We're interpolating Frame Y.

While rendering Frame X, before displaying it, we'll be working on the physics, AI, control input etc from Frame X-1. This would then be showing in the next frame, Frame Z.

Frame Y, to have this information, would then become a whole new frame. It would have to re-render everything based on what happened on Frame X. At that stage it's a a new frame. It's a dead frame, for lack of a better term. Display only, no interaction.

I can't see how you see the "new visual result 16.6ms earlier" unless Frame Y is using all the information calculated in Frame X. Which it could of course, but then it wouldn't work by interpolating from Frame X.

Maybe I'm missing something, but yes I think you're right to call that bull****. Would be incredibly happy to be proven wrong

[sj33]

I assumed this was all done in the post-process stage.

[Hohum]

I've just remembered why I stopped reading the framerate discussion thread a while back.

[Tig]

But then you'd be doing double the AI and physics calculations, which can't be the case.

Let's say usually we render Frame X and Frame Z. We're interpolating Frame Y.

While rendering Frame X, before displaying it, we'll be working on the physics, AI, control input etc from Frame X-1. This would then be showing in the next frame, Frame Z.

Frame Y, to have this information, would then become a whole new frame. It would have to re-render everything based on what happened on Frame X. At that stage it's a a new frame. It's a dead frame, for lack of a better term. Display only, no interaction.

I can't see how you see the "new visual result 16.6ms earlier" unless Frame Y is using all the information calculated in Frame X. Which it could of course, but then it wouldn't work by interpolating from Frame X.

Maybe I'm missing something, but yes I think you're right to call that bull****. Would be incredibly happy to be proven wrong

head asplode


I didn't consider that at all.
As you say, in order for a game to truly possess the smoothness of 60fps, the game logic - control sampling, AI etc - needs to be updated sixty times a second as well.
So I suppose this method would give you the appearance of a 60fps game, but with the responsiveness of a 30fps game.




That said... I still reckon the guy might be telling the truth about this reducing latency.
Say you've got frame 12 displaying on the screen right now. The game logic for frame 13 has been calculated, and the scene is about to be rendered again. At this moment the game can kick out a frame "12.5" and stick it on the screen - a frame which shows the intermediate (interpolated) state between frames 12 and 13.


If that's the case, I can see how it reduces latency slightly.
Although the gameplay is still essentially running at 30fps, you no longer have to wait for an entire frame to be rendered properly before the results of your input become visible.

[Matt]

Thing is, you'd have no rendering information with the newly processed game logic for frame "12.5". It's impossible without recalculating world space (which it would have done, as you'd use that in frame 13) but then also rendering it out. At that stage you're running a 60fps title.

Still a good idea though

[Tig]

Thing is, you'd have no rendering information with the newly processed game logic for frame "12.5". It's impossible without recalculating world space (which it would have done, as you'd use that in frame 13) but then also rendering it out.

I think you're mistaken on that last part.


It looks like they're using motion vectors to "distort" frame 12 into "frame 12.5".
So all they need is a fully rendered frame 12, and the motion vectors (velocity map) from frame 13. That's it - the intermediate image is generated there and then. No need to render frame 13 properly before it happens.

In the case of Andreev's demo, the depth buffer and velocity map for the next full frame are generated, but directly after this, midway through the processing, this data, combined with elements from the last frame, is used to interpolate the intermediate image before calculations on the next real frame continue.

anyways, I'm really hoping Lucasarts gives this out to other developers somehow. I reckon it would be at its best in racers.
I can't say I care much about Force Unleashed...

[Lyris]

Bring it on. I don't mind if there's a few artefacts, so long as the feeling of fluidity is there.

I'd love to do this on TVs, but like everyone's said, the lag makes it pointless.