More ergonomic asset loading to avoid "check if asset is loaded" boilerplate

[alice-i-cecile]

What problem does this solve or what need does it fill?

Assets need to be loaded before they can be delayed. However, with good reason, the asset loader does not block. In order to deal with this, you either need to wait until all of the assets are loaded or handle the possibility that your assets might not exist.

What solution would you like?

Bevy has a first-class built-in solution with minimal boilerplate. The state solution seems like a nice building block, but I'm not sure exactly what the correct API should be.

What alternative(s) have you considered?

State solution from examples.

Resource solution from cheatbook.

Both of these require a great deal of complex boilerplate.

Additional context

Async systems (#1393) might be the tool to solve this. Distill integration (#708) might also solve this.

[mockersf]

What would be interesting to know is why people want to wait for asset to be loaded?

• display a loading screen while it's loading? the state solution is perfect for that
• transform the asset once it's loaded? add a method on AssetServer to create an asset from an existing one #1665 can do that without blocking
• display the asset as soon as it's loaded? that should always be handled by the component that holds the asset handle, and bevy's built in components do it correctly

What are the other needs?

[kgv]

why people want to wait for asset to be loaded?

I describe the situation that I have.

I have a plugin that works with assets and needs to wait until the assets are loaded.
We have two types of assets:

• assets with data,
• assets with meta data that determines how the data assets are grouped.

The second type is a collection of handles for the first type.
The plugin needs to build a resource that depends on all asset of both the first and second types (asset's content).
In my case, state solution works great - i wait and change plugin's state.

But how the end user will check the plugin state?
He needs to create an additional system that will check the plugin state and change the game state accordingly.
Perhaps (this is just a guess) it would be more convenient to have combinators for states:

.add_system_set(SystemSet::on_update(GameState::Initialize `AND` PluginState::Done))

that should always be handled by the component that holds the asset handle, and bevy's built in components do it correctly

Does this mean that the plugin resource should return a Not Loaded error if you try to access it early?
@mockersf Could you link to this handling?

[mockersf]

We have two types of assets:

• assets with data,
• assets with meta data that determines how the data assets are grouped.

If I understand correctly, you are loading in your plugin directly the two types of asset, and once everything is loaded you bind them together according to the second type.

Ideally, your meta asset loader should handle itself loading the assets with data:

• load the meta asset, get the handle
  • the file will be loaded asynchronously
• in meta asset loader, load the meta asset file (we're asynchronous here)
  • parse how to organise the data assets, load their files, and get their handles. Organise everything with handles
    • those files will be loaded asynchronously
  • the meta asset is now holding handles to the data assets
• in data asset loader, load the data asset file (we're asynchronous here)
  • load the actual data
• to use your data, you have to wait for the meta asset to finish loading
  • after that, ideally you just use the handles to the data assets in your components
  • your plugin systems will be responsible to correctly handle the fact that the handle is not finished loading

to "bypass" the fact that you have to wait for the meta asset to finish loading to access a data asset that you know how to access through a stable identifier, there is the notion of labeled asset.

The Gltf loader is doing all that if you want to take a look.

Of course, if this is even possible really depends on how the assets you want to load are formatted...

Could you link to this handling?

For example rendering will ignore meshes that are not yet loaded:

bevy/crates/bevy_render/src/pipeline/render_pipelines.rs

Lines 97 to 101 in 45b2db7

	let mesh = if let Some(mesh) = meshes.get(mesh_handle) {
	mesh
	} else {
	continue;
	};

For assets that holds themselves an Handle (for example, Gltf can have handles to StandardMaterial, Mesh, ...) as the first level of Handle isn't used before it's loaded, and transitively for the handles it contains

[julhe]

What would be interesting to know is why people want to wait for asset to be loaded?

* display a loading screen while it's loading? the state solution is perfect for that

* transform the asset once it's loaded? #1665 can do that without blocking

* display the asset as soon as it's loaded? that should always be handled by the component that holds the asset handle, and bevy's built in components do it correctly

What are the other needs?

While I think the state approach is not bad, I can imagine that keeping track of your essential assets will be annoying and error prone. Imagine forgetting to add your ground collider mesh. On a slower systems they might be not present on-time and the player falls into the void, or even worse, only in 25% of the time.

I would suggest that every asset you load will be part of the essential group, unless explicitly overriden. You would check if your essentials has loaded by something similary like:

 if let LoadState::Loaded =
        asset_server.get_load_state() //name not final...
    {
        state.set_next(AppState::Finished).unwrap();
    }

Assets that are marked as non-esssential or can be streamed (textures, sounds, etc...) can be part of another group, so they don't delay the loading process to long. Something like asset_server.load_lazy("not_so_important_audio_file.ogg");

[mockersf]

the essential group

I like that! We would have to discuss the exact API/naming/handle handling... but it would certainly simplify something very common

[TheRawMeatball]

Here are some questions we need to answer:

• How does the user specify "essential assets"?
• How does the user specify which systems start immediately vs wait for essential assets?
  Is waiting for assets on a system opt in or opt out?
• How does a user extend this when they want to customize waiting behavior?

[mockersf]

• How does the user specify "essential assets"?

@julhe proposal is to have them essential by default and have an optout with load_lazy, I think offering an api like asset_server.load_in_group("myasset.png", MainGame); would be better and allow multiple group

• How does the user specify which systems start immediately vs wait for essential assets?
  Is waiting for assets on a system opt in or opt out?
• How does a user extend this when they want to customize waiting behavior?

This wouldn't handle waiting by default and wouldn't change what is currently available, it just adds a method asset_server.get_group_load_state(MainGame) (actually this method already exists and takes an Iter of handles). The user is still responsible for waiting with a run criteria, a state, ...

Another question is:

• does it keep strong or weak handle?
  • if strong, when are they removed from the assez server to allow cleanup?
  • if weak, the user still have to record all their handles in a struct somewhere so it doesn't change much

[julhe]

Another question is:

* does it keep strong or weak handle?
  
  * if strong, when are they removed from the assez server to allow cleanup?
  * if weak, the user still have to record all their handles in a struct somewhere so it doesn't change much

A reference counter should be fine! At the end of each frame, the asset system checks the reference count for each asset and frees up unused ones. (But keeping unused assets around in editor mode?)

• How does the user specify "essential assets"?

@julhe proposal is to have them essential by default and have an optout with load_lazy, I think offering an api like asset_server.load_in_group("myasset.png", MainGame); would be better and allow multiple group

• How does the user specify which systems start immediately vs wait for essential assets?
  Is waiting for assets on a system opt in or opt out?
• How does a user extend this when they want to customize waiting behavior?

This wouldn't handle waiting by default and wouldn't change what is currently available, it just adds a method asset_server.get_group_load_state(MainGame) (actually this method already exists and takes an Iter of handles). The user is still responsible for waiting with a run criteria, a state, ...

I think allowing for custom groups is overkill. You could get the same effect by bundeling components into scenes and wait for load completion on theses scenes instead. For now, I would make the API slim:

• asset_server.load() -> Handle<T> for most cases.
• asset_server.is_loading() -> bool you can check if all assets are ready or display a loadscreen. My idea is that if you use load() while the game is running, all simulation systems hold. Ugly, but neccessary to avoid any error state I mentiont earlier (missing colliders, etc.). Also avoidable by the user.
• asset_server.load_lazy() -> Handle<T> for assets you need soon (like an upcomming cutscene). Doesn't block the simulation.

[alice-i-cecile]

@NiklasEi has made a community crate to address many of these pain points: https://github.com/NiklasEi/bevy_asset_loader

I'm going to experiment with it some more, but I think this is an excellent candidate for integration into the engine itself once it matures a bit more.

[cart]

Yeah I think it makes that particular pattern much nicer. But before adding a new concept / high level abstraction like AssetCollection, I'd like to see how things like "async system + async asset loading" work out in terms of ergonomics. Ideally we can have lower level primitives that interact with each other ergonomically and allow developers to compose the desired behavior without the need to provide higher level abstractions on top.

But if we can't make that work well enough I think integrating something like AssetCollection is probably the right call.

[alice-i-cecile]

A new user tried to implement this functionality for our Breakout example game as part of #2094. Here's what they found:

as far as I understand, there's no (Bevy) idiomatic way of accomplish this. I've thought of 3 possibilities:

1. implement states ("loading" and "game"), with the assets loaded on the loading state, which then would switch to the game state once completed; I think this is something very interesting to put on a showcase example. unfortunately (as far as I understand) this is incompatible with the fixed time step; there are hacks to work this around, but I suppose that's not something one would put in an official example (I suppose that additionally, breakout is one of the first/most commonly looked examples)
2. implement assets loading as a setup system with a (ugh!) while loop. this works, but I guess it's very unidiomatic
3. I've seen that Bevy had AssetServer#load_sync on v0.1, which I guess does a synchronous load, but it's not present anymore on 0.5.

Based on the above, it seems that Bevy's not ready for a clean implementation of this functionality. But of course, if you have any indication, I'm happy to implement it :slight_smile:

The gist of it is that #1295 is still frustrating, blocking the most natural solution.

[alice-i-cecile]

For now, we're going to proceed with a synchronous blocking loading solution, using a loop that sleeps to avoid eating all of the user's compute.

[64kramsystem]

Hi there! I'm the user who tried to implement the above.

First, the use case, in relation to this:

What would be interesting to know is why people want to wait for asset to be loaded?
[...]
What are the other needs?

Error checking :) I may be missing something here, but in the breakout example, if an asset load fails, the game proceeds, without text, and with a warning in the console.

For reference, the related code is this:

TextStyle {
    font: asset_server.load("fonts/FiraSans-Bold.ttf"),
    font_size: 40.0,
    color: Color::rgb(0.5, 0.5, 1.0),
},

I don't see any direct way to control error cases. Maybe there are ways to handle this, but I think that error handling is a legitimate case.

Regarding the solutions:

1. implement states ("loading" and "game"), with the assets loaded on the loading state [...]
2. implement assets loading as a setup system with a (ugh!) while loop. this works, but I guess it's very unidiomatic [...]
3. I've seen that Bevy had AssetServer#load_sync on v0.1, which I guess does a synchronous load, but it's not present anymore on 0.5.

I was wrong (sorry 😬). Solution #2 does not work. As per @TheRawMeatball suspicion, loading assets needs another system to do the work, so blocking the system in a loop will actually not exit, and block the whole game. So, it's even worse 😬

@NiklasEi has made a community crate to address many of these pain points: https://github.com/NiklasEi/bevy_asset_loader

Based on a quick look, I think this uses states, which is my favourite solution, but AFAIK makes it incompatible with FixedTimeStep without trickery.

It'd be interesting to see the trickery that the Bevy Cheatbook mentions.

[mockersf]

Error checking

The idiomatic way would be with states... which may be hard to get to work with a fixed time steps.

You could go the old way to do states:

• have a resource that hold your state value
• start loading your assets, set your resource state to Loading
• have a system that run every frame listening to asset loading event
  • on success, change the resource state to Game
  • on failure, do... whatever you do with asset loading error
• in your game play systems that run on a fixed time steps, look at your resource state and exit if it's not Game

[alice-i-cecile]

@saveriomiroddi there might actually be a good way to do this: impl FromWorld for the assets, and then use app.init_resource. If I understand correctly, this should result in synchronous loading ahead of time and is still idiomatic.

[mockersf]

@alice-i-cecile How? You can't load assets synchronously thought the AssetServer.
You could load them manually synchronously, but synchronous means blocking. If you do it during the init phase, it will block either before opening the window, or with an empty window, that doesn't seem something we want?

[alice-i-cecile]

Oh wait, no, you're right there. AssetServer will never block. Hmm.

[64kramsystem]

Error checking

The idiomatic way would be with states... which may be hard to get to work with a fixed time steps.

You could go the old way to do states:

* have a resource that hold your state value

* start loading your assets, set your resource state to `Loading`

* have a system that run every frame listening to asset loading event
  
  * on success, change the resource state to `Game`
  * on failure, do... whatever you do with asset loading error

* in your game play systems that run on a fixed time steps, look at your resource state and exit if it's not `Game`

Phew 😅 I think I won't implement (propose) this in the breakout, though. Unfortunately, the asset loading is part of a setup system (scoreboard), so implementing this strategy (which seems the only one 😱) would require moving it to the main (as in "running game") systems, which I think is an odd design, at least, something I wouldn't put in an official example/tutorial.

I think it'd be good to put in the Cheat Book, though. I'll open a PR once my current PR on the same subject is merged.

Ultimately, I think the root of the problem is having a FixedTimeStep compatible with states, which is something that @TheRawMeatball at some point will bake 🍰 😄.

[alice-i-cecile]

bevy_asset_loader by @NiklasEi has a great write-up of the approach they're using in that crate. Once that's matured a bit more I think we should seriously consider upstreaming much of that approach: it's a serious win.

Of course, it probably also makes sense to see where states end up...