Palico-GameProgrammer

Asset Management

In this blog, I will show how my assets are managed and connected in the game. Prefab is also a crucial feature in game development so I will describe how I implemented it.

1. Meta Generator

Before actually stepping into the engine, we need a third project in the solution called MetaGenerator along with the GameEngine and GameProject. Like what Unity did, the meta file will handle all the assets’ link. If you delete one of the .meta file, all the previous link will just break and the engine will treat it as a new asset generating a brand new .meta file for it.

Meta File

Basically, my meta file is just a Json file and saves the data as follow:

{
  "Asset" : "Textures/Button.png",
  "FileName" : "Button.png",
  "Guid" : "f8dd170e-6a86-46f1-a304-a4303a78233e",
  "Location" : "Project",
  "Type" : "TextureAsset"
}

“Asset” is the relate path of it. Location shows if it’s a engine asset or a project, which has different path. (More information about file path please check the File Manager Blog)

Noted that all these contents are generated automatically. That’s the reason why I would like to create the generator. When you want to add a new asset, such as a texture, just drag and drop it into the asset folder. The solution will run the generator first to check any new assets and generate the .meta file for it.

Generate

void Generate(fs::path path,AssetLocation loc)
{
	fs::path assetsDir = path;

	if (!fs::exists(assetsDir))
	{
		std::cerr << "Assets folder not found!\n";
		return ;
	}
	for (const auto& entry : fs::recursive_directory_iterator(assetsDir))
	{
		if (entry.is_regular_file())
		{
			fs::path filePath = entry.path();

			if (filePath.extension() == ".meta")
			{
				if (!fs::exists(filePath.replace_extension("")))
				{
					std::error_code ec;
					fs::remove(entry.path(), ec);
					if (!ec)
					{
						std::cout << "Deleted: " << entry.path() << "\n";
					}
				}
			}
			else if(filePath.extension() != ".json")
			{
				fs::path metaPath = filePath;
				metaPath += ".meta";

				if (fs::exists(metaPath))
				{
					continue;
				}

				CreateMetaFile_(filePath,path, loc);
			}
		}
	}
}

This is the core function of the generator. It will recursively check all the files in the asset folder.

When it find a .meta file without the source file - the asset, it means the user has deleted the asset file. So it will delete the .meta file automatically.

And if it find both the source file and the .meta file, it will just pass it which is not a new asset.

Finally if it not match all the conditions above, it will call the CreateMetaFile function.

Create File

void CreateMetaFile_(fs::path assetPath,fs::path folderPath,AssetLocation location)
{
	fs::path metaPath = assetPath;
	metaPath += ".meta";

	json::JSON j;
	j["Type"] = GetAssetType(assetPath);

	if (location == AssetLocation::Engine)
	{
		j["Location"] = "Engine";
		j["FileName"] = assetPath.stem().string();
	}
	else
	{
		j["Location"] = "Project";
		j["FileName"] = assetPath.filename().string();
	}

	UUID _uid;
	CreateUUID(&_uid);

	j["Guid"] = GUIDTostring(_uid);
	fs::path relativePath = fs::relative(assetPath, folderPath);

	std::string pathStr = relativePath.string();
	std::replace(pathStr.begin(), pathStr.end(), '\\', '/'); 

	j["Asset"] = pathStr;

	std::ofstream metaFile(metaPath);
	if (!metaFile)
	{
		std::cerr << "Failed to create: " << metaPath << "\n";
		return;
	}

	metaFile << j.dump()<<std::endl;
	std::cout << "Generated: " << metaPath << "\n";
}

Here is all the contents actual generated.

The type is decided by the .extension of the asset file, using a static map:

std::string GetAssetType(fs::path assetPath)
{
	static const std::unordered_map<std::string, std::string> extensionToType = {
		{".png", "TextureAsset"},
		{".jpg", "TextureAsset"},
		{".wav", "SoundAsset"},
		{".ttf","FontAsset"},
		{".prefab","PrefabAsset"},
		{".scene","Scene"},
	};

	auto it = extensionToType.find(assetPath.extension().string());
	if (it != extensionToType.end())
		return it->second;
	return "Unknown";
}

The Guid is generated by a third party website: https://www.uuidgenerator.net/

In the end, all the Generate function in the main function.

int main()
{
	Generate(ASSET_DIR,AssetLocation::Project);
	Generate(ENGINEASSET_DIR,AssetLocation::Engine);
}

2. Asset

Next, let’s talk about the asset itself.

class Asset : public Object
{
	DECLARE_ABSTRACT_DERIVED_CLASS(Asset, Object)
public:
	std::string& GetFileName() { return fileName; }
private:
	friend class AssetManager;
	virtual void Load(json::JSON j, std::string& _fileName) = 0;
	virtual void Destroy() = 0;
protected:
    int refCount = 0;
	std::string fileName = "";
};

This is the parent class of all type of assets, and each type need to implement its Load and Destroy.

Now, it has 4 types of asset: Texture, Font, Sound and Prefab.

Texture

void TextureAsset::Load(json::JSON j, std::string& _fileName)
{
	fileName = _fileName;
	fs::path path = FileManager::GetAssetPath(j);

	SDL_Surface* surface = IMG_Load(path.generic_string().c_str());
	if (surface == nullptr)
	{
		Debug::Warning("Cannot find Texture!" + path.generic_string());
		return;
	}

	SDL_Renderer* renderer = RenderSystem::Instance().GetRenderer();
    texture = SDL_CreateTextureFromSurface(renderer, surface);

    SDL_FreeSurface(surface);

    if (!texture) {
		Debug::Warning("Cannot Create Texture!" + path.generic_string());
		return;
    }

	SDL_SetTextureBlendMode(texture, SDL_BLENDMODE_BLEND);
}
void TextureAsset::Destroy()
{
	SDL_DestroyTexture(texture);
	texture = nullptr;
}

When we want to use a texture, just call GetTexture function. It will return the loaded texture;

SDL_Texture* GetTexture() { return texture; }

Font

The font asset is quite same as the texture. The only different is the size. Whenever we change the size, we need to load it again from the source.

void FontAsset::Destroy()
{
    for (auto& pair : fontMap)
    {
		TTF_CloseFont(pair.second);

        pair.second = nullptr;
    }
	fontMap.clear();
    TTF_Quit();
}

void FontAsset::Init()
{
    if (TTF_Init() == -1) {
        std::cerr << "SDL_ttf could not initialize! TTF_Error: " << TTF_GetError() << std::endl;
    }
}

void FontAsset::LoadFont(json::JSON j, int _size)
{
    fs::path path = FileManager::GetAssetPath(j);

	TTF_Font* font = TTF_OpenFont(path.generic_string().c_str(), _size);
	if (font == NULL)
	{
		std::cerr << "Failed to load font! TTF_Error: " << TTF_GetError() << std::endl;
		TTF_Quit();
	}
    fontMap.emplace(_size, font);
}

void FontAsset::Load(json::JSON j, std::string& _fileName)
{
    fileName = _fileName;
    Init();
    LoadFont(j,fontSize);   
    path = j;
}

void FontAsset::SetFontSize(int _size)
{
    if (fontSize == _size) return;
    
    LoadFont(path,_size);
    fontSize = _size;
}

The prefab will be talked later in a separate part.

3. Asset Manager

So how the asset manager works?

First of all, it will load all the .meta file, not the asset, only the .meta file. It just like a database storing all the .meta file and load the specific asset when a scene asked.

void AssetManager::GenerateMetaDB()
{
	auto filePath = FileManager::GetALLMetaFiles(FileManager::GetAssetFolderPath());
	for (auto& p : filePath)
	{
		auto j = FileManager::LoadJson(p.c_str());
		std::string guid = FileManager::JsonReadString(j, "Guid");
		std::string fileName = FileManager::JsonReadString(j, "FileName");
		metaDatabase.emplace(GetHashCode(fileName.c_str()), p);
	}
}

It will put all the file name hash code and the path into a map.

Noted that if there is an editor, the better way is using the guid as the key, path as the value. But currently we don’t have an editor and I don’t want to copy paste the guid manually. You will see what I mean.

After that, it will load the engine asset as the default asset:

void AssetManager::LoadEngineAsset()
{
	auto filePath = FileManager::GetALLMetaFiles(FileManager::GetEngineAssetFolderPath());
	for (auto& p : filePath)
	{
		auto j = FileManager::LoadJson(p.c_str());
		std::string type = FileManager::JsonReadString(j, "Type");
		std::string fileName = FileManager::JsonReadString(j, "FileName");

		metaDatabase.emplace(GetHashCode(fileName.c_str()), p);
		if (type == "Scene")
			continue;

		Asset* asset = (Asset*)CreateObject(type.c_str());
		asset->Load(j, fileName);
		engineAssets.emplace(fileName, asset);
	}
}

These two function will be called in the Initialize function, no matter there is a scene or not.

Load

Then, all the actual assets will be loaded when loading a scene.

My scene structure is like this:

{
	"assets": [
		"Button.png",
		"player.png",
		"cour.ttf",
		"player.prefab",
        //...
	],
	"scene": {
		//...
	},
	"ui": {
		//...
	}
}

Generally, it’s saved as a json file. There are 3 parts: assets, which shows all the necessary asset in the scene. scene, saving all the entity and prefabs in the scene. UI: the ui for this scene.

So when I load a scene, I will know exactly which assets are needed.

void AssetManager::Load(json::JSON& _json)
{
	if (!_json.hasKey("assets"))
	{
		Debug::Warning("No Assets Loaded!");
		return;
	}

	json::JSON metaArray = _json["assets"];

	for (auto& meta : metaArray.ArrayRange())
	{
		std::string fileName = meta.ToString();
		std::string path = metaDatabase[GetHashCode(fileName.c_str())];

		if (assets.count(GetHashCode(fileName.c_str())))
		{
			assets[GetHashCode(fileName.c_str())]->refCount++;
		}
		else
		{
			json::JSON j = FileManager::LoadJson(path.c_str());
			Asset* newAsset = (Asset*)CreateObject(FileManager::JsonReadString(j, "Type").c_str());
			newAsset->Load(j, fileName);
			newAsset->refCount = 1;
			assets.emplace(GetHashCode(fileName.c_str()), newAsset);
		}
	}
}

So here I’m using the File Name as the link in a scene and the asset. But the more professional way is using the guid. But as I mentioned before, there is no editor yet, so the file name is easier to handle manually.

Because there may have multiple scenes loaded, the same asset don’t need to be loaded twice. That’s reason why I added a refCount in the asset class. If the same asset are found in the map, just increase the refCount.

Unload

And when I want to unload a scene, do not delete the asset directly. Reduce the refCount and if it’s 0, then delete it.

void AssetManager::Unload(json::JSON& _json)
{
	if (!_json.hasKey("assets"))
	{
		return; 
	}

	json::JSON metaArray = _json["assets"];

	for (auto& meta : metaArray.ArrayRange())
	{
		std::string fileName = meta.ToString();

		unsigned int hash = GetHashCode(fileName.c_str());

		auto it = assets.find(hash);
		if (it != assets.end())
		{
			it->second->refCount--;

			if (it->second->refCount <= 0)
			{
				delete it->second; 
				assets.erase(it);  
			}
		}
		else
		{
			Debug::Warning("Attempting to unload an asset that isn't loaded: " + fileName);
		}
	}
}

Acquire

In the gameplay logic, sometimes we need to load an asset for example spawn an prefab. Here is how can we get a asset.

template <typename T>
T* GetAsset(const std::string& fileName)
{
    Asset* rawAsset = GetAssetInternal(fileName);
    if (!rawAsset) return GetDefaultAsset<T>();
    return (T*)(rawAsset);
}

template <typename T>
T* GetEngineAsset(const std::string& fileName)
{
    Asset* rawAsset = GetEngineAssetInternal(fileName);
    if (!rawAsset) return nullptr;
    return (T*)(rawAsset);
}

There is two template functions provided for getting an asset. It will call the internal function to get the asset.

Asset* AssetManager::GetAssetInternal(const std::string& fileName)
{
	STRCODE key = GetHashCode(fileName.c_str());
	auto it = assets.find(key);

	if (it != assets.end())
	{
		return it->second;
	}
	Debug::Warning("CANNOT find asset!!! " + fileName);
	return nullptr;
}

So when I need an asset, just call the function. Here is an example in my gameplay implementation:

std::string bulletName = FileManager::JsonReadString(jsonData, "bullet");
bullet = AssetManager::Instance().GetAsset<PrefabAsset>(bulletName);

4. Prefab

Next, let’s talk about the prefab!

Generally speaking, a prefab is just an entity with several components. But there are still some difference: First, a prefab don’t need to appear in the scene. It just an asset. Second, it’s a blueprint that you can duplicate the entity base on the template.

Structure & Load

{
	"type": "Prefab",
	"name": "Player",
	"id": "0000-0000-0002",
	"tags":[
		"Player",
		"DontDestroyOnLoad"
	],
	"components": [
		{
			"type": "Transform",
			"position": {
				"x": 0.0,
				"y": 0.0
			},
			"scale": {
				"x": 1.0,
				"y": 1.0
			},
			"rotation": 0.0
		},
		{
			"type": "CircleCollider",
			"isRendered": false,
			"center": {
				"x": 0.0,
				"y": 0.0
			},
			"layer": 0,
			"isTrigger": false,
			"isStatic": false,
			"size": {
				"x": 100.0,
				"y": 100.0
			},
			"radius": 24.0
		},
		{
			"type": "PlayerController"
		},
		{
			"type": "Sprite",
			"name": "Sprite01",
			"asset": "player.png",
			"layer": 0,
			"size": {
				"x": 48.0,
				"y": 48.0
			},
			"offset": {
				"x": 0.0,
				"y": 0.0
			},
			"rotation": 0.0,
			"flip": "SDL_FLIP_NONE",
			"color": {
				"r": 1.0,
				"g": 1.0,
				"b": 1.0,
				"a": 1.0
			}
		}
	]
}

There is the structure of my player prefab. It is totally the same as the entity.

When I want to load the prefab, It will create an entity, and load all the data for me.

void PrefabAsset::Load(json::JSON j, std::string& _fileName)
{
    fileName = _fileName;
    fs::path path = FileManager::GetAssetPath(j);

    json::JSON prefabJson = FileManager::LoadJson(path.generic_string().c_str());
    prefab = (Entity*)CreateObject("Entity");
    prefab->Load(prefabJson);
}

In the entity load function, it will load every necessary information and create components for it. Noted that I didn’t add it in to the scene after loading. It just an asset cached in the asset manager.

Spawn

So here is the question: How can I spawn it? If I just copy the entity itself, it will have the exact same data. But remember that, all the components are pointers in the entity. So it will only copy the address of them not creating new components. Instead of shadow copy, I want deep copy everything.

Entity* Entity::Clone()
{
	Entity* cloneEntity = (Entity*)CreateObject("Entity");
	for (auto& c : components)
	{
		Component* cloneComponent = c->Clone();
		cloneEntity->components.push_back(cloneComponent);
		cloneComponent->owner = cloneEntity;
		if (cloneComponent->GetDerivedTypeClassHashCode() == Transform::GetTypeClassHashCode())
		{
			cloneEntity->transform = (Transform*)cloneComponent;
		}
	}
	cloneEntity->tags = tags;
	cloneEntity->name = name;
	return cloneEntity;
}

Here is the Clone function in the Entity class. I manually create a new entity, save other data and traverse all the components.

In the Component class, Clone is a empty virtual function. Children class will decide what they need to clone. But here I don’t want to manually write the clone function every time when I add a new component. We can just copy that component’s memory, so that it will contains all the data.

Component* Transform::Clone()
{
	Transform* clone = (Transform*)CreateObject("Transform");

	*clone = *this;

	clone->owner = nullptr;
	return clone;
}

Every components’ clone function is quite same instead of the class name. So I created a macro to do the same things for me.

#define CLONEABLE_IMPLEMENT(ClassName) \
Component* ClassName::Clone() \
{ \
    ClassName* clone = (ClassName*)CreateObject(#ClassName); \
    *clone = *this; \
    clone->owner = nullptr; \
    return clone; \
}

So now, we only need to add the macro when creating a new component

CLONEABLE_IMPLEMENT(Transform)

I created a namespace called Gameplay to handle the Spawn and Destroy.

Entity* Gameplay::Spawn(PrefabAsset* prefab)
{
	Entity* e = prefab->GetPrefab();
	Entity* res = e->Clone();
	SceneManager::Instance().GetCurrentScene()->AddEntity(res);
	res->Initialize();
	Debug::Log("Spawn prefab: "+res->name);
	return res;
}

Here is the spawn function. It will clone the entity and add to the current scene. And because of the global namespace, I can just call it wherever I need:

Entity* e = Gameplay::Spawn(bulletAsset);