Hi all. I’m a Unreal user and working on a cinematic rhythm game with quick time events. I’m planning to simply play a video and if the input is incorrect it swaps to the “wrong” video for a beat then swaps back to normal cinematic. Like Dragon’s Lair.
The mini games / songs will be mostly marketing content and I want to distribute the game as wide as possible, hence me considering Defold that seems attractive in terms of lightness and cross-platform.
I only need to play 2 videos, check input at specific times and have a gui. I’ve seen there’s a video player extension that seems a bit limited but I want your opinions if this game is feasible in Defold. And if I can play videos, can I also stream them for the web version?
Thanks! Mostly normal playback of 2 videos at the same time. Detect input while they play and swap video versions depending on input.
There will be audio but it can be played by another component.
Streaming videos is what I thought of for web version since video filed are big. But that’s secondary, local video playing is what I need most. And seeking if available but I saw it’s not implemented.
I recently added a basic mpeg1 video decoder as an alternative to the much more complex one that we also provide. You can try it here:
The extension currently only supports playing one video at a time, but that can easily be changed. The extension supports seeking. It can be made to also support streaming.
You recommend using this one? What’s the advantage of mp1 over mp4? It’s less cpu intensive? And how can I change it to be able to play 2 videos at the same time?
Small remark : I thought that playing 2 videos at the same time would allow a more seamless swapping between them. We only ever see one video full screen. Maybe playing one video then replacing it with the second one at a precise time can be seamless? In this case I don’t need to play 2 videos but just display them at different seek times. Do you think it would work?
Thanks for the help. If the swap is seamless then no need for 2 videos at the same time ! I need to test and for that I’ll have to get a bit familiar with the engine.
The extension already supports streaming or does it need to be implemented?
Hello @britzl , I did some testing for video playing and swapping on mobile and I have the following questions please :
I coded a basic video swapping when I tap anywhere on the phone screen. It’s instant on desktop but there’s a small lag on mobile (Redmi 7A). The video size is 720x1440, I don’t have any more lag when I reduce the video to 360x720 but it’s very pixelated.
Is there a way to load a second video and make it visible on tap ? That’s why I wanted to play 2 videos at the same time but using open() + mpeg.seek() without optimisation works already quite well so maybe there’s a simpler solution. Any suggestion on how I could make it instant on low-end phones ?
the mpeg player is supposed to play audio ? I can set audio to true in the code but there’s no audio at all.
the video playing seems more pixelated in Defold than when I play it on a normal video player in windows. Is there some kind of anti-aliasing used in desktop video players or is it a Defold thing ?
I’m using a cumulated delta time as argument for seek(). It works fine but still have imprecise video frame selection, resulting in small hitches. Is there a way to get the active frame of the video playing and convert it to time so that I can use it as a more precise time argument for seek() ?
I don’t understand why we need to divide the screen width by the logo size (screen_width / logosize) in order to get the video size. Could you please explain the logic behind it ?
And as a reminder the video playback crashes after a bit of time. The bug is reported on github and I added some additional behavior info, for example I can play the full video when I keep swapping between 2 videos at frequent intervals.
Sorry for the many questions. The whole gameplay relies on stable video playing so it’s a bit different from the usual gameplay questions. Thanks a lot !
The extension need to be reworked to support multiple video instances. This is a fairly small change. Please create a feature request!
No. This is a bit more complicated to achieve. Again, please create a feature request!
Not sure of the reason. It could be in the decoder that is used in Defold. Or it could be some kind of smoothing that is done in other video players. It might be improved through a shader on the video quad/sprite.
There is an option to for more precise seek, which I set to false:
We could make this an argument to the seek() function. Please open a feature request!
This is done so that the video will be stretched to span the entire screen. If you don’t scale the sprite, the video will be played in the 128x128 pixels of the sprite only.
I’ve seen the bug report and I will look into it soon. I’m currently busy with some other tasks.
For the multiple video instances + precise seek, I’m not sure these will have a positive impact on performance on low-end devices. I don’t want to add feature requests that will add work for you without a garanteed performance gain.
What is your suggestion on that ? What’s the surest way to have a seamless swap on low-end phones ? Are 2 instances of video players better or worse for performance ? Keep in mind that the lag is pretty small and doesn’t happen everytime on my low-end phone, so we’re not far from seamless already.
I need to do some additional tests before asking for a precise seek, maybe I need to convert the delta time to a value closer to the video fps (24fps). I’ll try other approaches before deciding on it.
The real issue for me is the anti-aliasing. Is it because it’s not applied to the texture ? Is MSAA applied by default ? What kind of shader do you think about ? Can give me some keywords to search for ? I’m not very advanced on shaders.
For me it’s the best solution because a good small video would solve all lag issues, hence no additional features needed.
About the resizing of the 128px sprite, does it mean I can create a sprite that’s 360x720 and display the video on it without resizing ? What’s the recommended practice ? Should I use a quad ? Can it be related to the low quality of the video mentioned before ?
Ok for the audio, I’ll play it separately with the audio component.
I think it is valuable to be able to play more than one instance at a time. When I made the initial implementation I took the quickest path to something useful, and that was a single instance only. Open a feature request if you feel the need for multi instance playback!
Adding an option to do precise seek is trivial.
I honestly don’t know either. I’d have to do some research of my own. A quick search gives me this:
Ok in this case how I see it is to have 2 mpeg instantiated and buffering at the same time then show one or the other. I’ll put a feature request for instancing, precise seek and streaming from a server.
Tfhank for the shader links. I’ll check it out.
Ok I’ll try both quad and sprite and tell you how it goes.
@britzl I opened the features requests on the extension github. I couldn’t flag them as a feature requests but only issues, I guess it’s you who needs to do it ? Thanks.
Hi @britzl . Thank you very much for the extension updates!
I saw that you updated the Readme usage. The handle in the example is what allows to have multiple instances right? For example I can have handle1 and handle2?
I don’t understand the usage of mpeg.getframe(), in the Readme, you decode handle and not the buffer from getframe()?
Thanks again! I’ll download the new version tomorrow and test it.
Correct, it’s basically pointer to the structure holding the video info. You need to make sure to call mpeg.close(handle) when you no longer need it, otherwise you’ll have a memory leak on your hand.
It works like this:
mpeg.decode(handle, dt) will advance the internal playback timer by dt and decode video up to this time.
mpeg.get_frame(handle) will return a buffer with the bytes where the decoded frame data (pixels) is stored. You can use the buffer as a texture on a sprite for instance.
Hi @britzl , I worked on the video swapping with the updated repo and done a few tests :
video instances and precise seek seem to be working but it’s hard to test because the video crashes after 50 frames now. I hope this bug isn’t too hard to solve, right now I can’t do much because of this crash and it’s the most pressing issue.
video streaming works better than the play(), it doesn’t crash but the video resets after 10s. It seems to be wrong about the video length. Should I file a bug report ?
does video streaming allow multiple instances ? I didn’t explore too much because I have a hard time understanding the code of the streaming function. Thank you for putting a sample code because it’s a bit beyond me for now.
