Before the end

I changed the orthographic camera to perspective as you can see in the above video, we decided to do this because the sense of depth was necessary and we might need to move the camera closer to the character in places like caves.

I also created multiple arrays of vertices/tris/normals/etc which will act as buffers for when I'm generating a mesh from chunks, this will reduce the amount of garbage that would have otherwise produced by the temporary mesh arrays, the only downside with these buffers is that they will use up a small amount of memory. Of course this could be avoided if I could just create a single buffer of each vertices/tris/etc and give it to unity's mesh class with a length parameter but I can't.

PS: Happy new year!

Designing, optimizing and moving forward

The new cube creator is finished for now, I will be adding more options and features to it later.

I haven't mentioned anything about greedy meshing and there's a good reason for that, you might have noticed that there are several different textures near each other making the greedy meshing very small and not all that useful, the other problem is that I have all of the textures in one atlas so repeating a certain texture would be problematic because you can't overflow the uv coordinates in an atlas so that the texture would repeat, so I had to created a shader which could repeat a texture inside an atlas by adding a certain amount, let's say 512, to the uv coordinates which would tell the shader how many times the texture would repeat, after that it would be just a few calculations to determine the current texture index by just dividing the uv by 512 after which we can get the actual uv coordinates for the certain part in the mesh.

While we can now repeat a certain texture it still does not remove the problem that we have multiple textures near each other, for that I would just need to create a new atlas and combine textures in a 2 * 2 texture in every possible combination so I can create 2 * 2 * 2 size cubes with one texture that has 4 different texture combinations for each side instead of 1 * 1 * 1 with one texture for each side.

There might be better ways to do this but I can't dwell for too long in optimizing the terrain generator or it might become a never ending project.

Me and Samuli have been designing crafting for the game and an interface for it and it looks great.

A mockup of the crafting interface.

I've also been creating a house editor for the game, we are going to make preset assets for NPCs, players can create custom houses in the game but they can also use presets as well. A house will be created in 3 different stages, first a player will place walls for the house after which he can place the floor or the roof or even more layers, after the house is designed players can predetermine what furniture the house will have before building it.

Terrain facelift in progress part 2

I've almost finished the new cube creator, only a few features are missing from the biome editor, but already we have made some very interesting and good looking terrains.

As you can see in the above image, we can edit the size and heights as well as what cubes will be generated to different heights, I also got the sub-biomes working.

Above image is a sub-biome of grasslands.

Desert biome and its sub-biome

Samuli also finished a new version of the 3D character model which is in the game already

We are continuing on adding more biomes and I'm leaning towards working on the gameplay mechanics.

Terrain facelift in progress

I've been recreating the cube creator as it lacked some features and it occasionally gave unity some problems as it started reporting errors of null objects being synchronized or something along those lines, anyway the new unfinished cube creator will make the map generation so much more flexible as the new features to the biomes and the new sub-biomes will allow broader range of options to the map generator, with it I can generate much more realistic and unique terrain.

As I mentioned sub-biomes adding them will require a new map, like the temperature and humidity maps, the new map will have numbers going from 0 to 100, so depending what range the sub-biomes will have, it will affect which sub or parent biome will be selected.

I mentioned in the previous post that I optimized the generation of the heightmap, thanks to that optimization and the lake algorithm I was able to sped up the isometric image rendering which I can now use in the cube creator to view different landscapes faster.

I also finally added the 3D model of the character to the game as well as tree felling.

Me and Samuli been talking and designing quests for the game, nothing big just some examples, we want to have dynamic quests that are build from multiple different quest pieces, we've also been trying to make the quests so that there are multiple ways to complete an objective, the player can decide how he would complete said objective, we would only give a possible direction from which the player could complete the objective.

Seemingly impossible lake part 2

I ran to a problem with the lake algorithm, as I create it by using a heightmap and the problem with this was that holes in the ground will not be filled, of course I though that just creating a floor under the lake and walls at the border will fix this but there was one special case to which this "fix" did not work very well, here's a picture

As you can see there is a huge hole here and the reason for this is that above there is a cliff which makes it the highest elevation in the heightmap which in turn makes this side of the mountain not being filled, thankfully fixing this was quite simple and effective but it does not follow the wall perfectly as it will convert some blocks to water making the wall a little more linear.

Other than this problem, I've optimized the lake generation as well as the creation of heightmap.

Oh and I did try to do something about the rendering of chunks as the useless garbage allocation is insane, but the mesh creation in unity blows and I can't actually prevent useless garbage allocations, but I did make it a little faster and use less memory

Seemingly impossible lake

I've finished adding water to the game sort of, I can generate lakes and render them but there are no "fluid simulation" happening, yet, and there are still a lot of things that I have to think through before I can officially say it's finished.

So how did I add water to the game, it was actually quite difficult as the terrain is easy to shape but adding something like liquid is harder as it can spill, overflow and so on, so what I first did was
I created a heightmap from the coarse noise map from which I searched spots where a single water could be placed without spilling, after I had done that I made a sort of a filling algorithm that would raise water to nearby neighbors until a x number of nodes were found, then I remove all nodes that have neighbors (except ones that have been found) that are lower in height than the highest node that was found, now I had possible lakes that could be formed but there was another step I had to do.

Left has water locations, middle one lakes and right is the heightmap.

Next up I created a heightmap from the finer noise map, now that I had all the lakes from previously, I go through all of the lakes and find the lowest point of each lake and I locate them in the finer noise map from which again use a filling algorithm, this algorithm is essentially a better version that I used to find the lakes. The result does not fully represent the lake that was found from the coarse map as the finer noise is modified slightly to add slight differences to the terrain.

I can't create all the lakes in one go as it takes quite a bit of resources and space from the harddrive but thanks to the coarse map I have all the potential lakes from which I can actually generate them from.

Below is a gif of a lake, not the same one as in the heightmap.

View post on imgur.com

S-...Sleep

Well I've finished creating the template for Foraging, there is a chance to gain items from foraging and you can level it up as you are doing it, leveling foraging for now gives you a better chance to get items, later I might add different foraging types such as finding weapons/armors or other valuables beside food and resources.

I've also been programming the inventory and character GUI which has taken most of my time, currently the inventory slot functionality is mostly done, some item splitting and such features are missing as well as using items, the equipment side is not done. In the character side I've finished the needs, health and mana but still working on the skills interface.

Programming the character saving is becoming quite the tedious task, I have to split a characters information to several files, if a character is not important it's useless to save such as inventory or skills because they can be generated by a seed. Splitting the inventory and skills to different files also helps me in adding more skills and other data as I've yet to design and define them because all of the characters data is converted to bytes so I have to carefully "map" each variable in the save file.

PS: Oh and I've been cleaning for 3 days straight with little sleep while friends are playing DnD? thanks.

Slow progress

I've been sick for a week then struck a heatwave, I hate high temperatures and working without any kind of air conditioning is horrible, so I've only been working on memory usage and trying to eradicate all garbage, other than that it's been quite quiet.

Oh I have been searching for more ways to improve the terrain generation and looking for ways to add water and there has been some success although I've yet to implement any (except for one image below of random generated canyons) but it's looking promising.

Gameplay and more terrain generation

So I've more or less finished the refactoring and I've begun adding some gameplay elements starting with foraging items from the ground and proceeding from there to some basic survival mechanics, crafting and building, after that I'm adding some neutral and hostile npcs with which the player can interact with. I'm also trying out some different terrain types to see which I should use and so on.

Samuli also has modeled some house parts for the game and I'll be adding them once I have programmed building.

PS: I'm still updating the task list at Onwards

Latent optimizations

So I've been refactoring the project and changing most of the integers to long so I can generate larger maps is going well, there were some hiccups here and there but it's still an ongoing process.

I also noticed some potential optimizations in the chunk rendering part which I've now added, I noticed these because I changed the cube array in the chunk to the length of 4096 from 2048 to take advantage of the previous optimizations (which I mentioned here Rivers, streams and more optimizations), the change was a double edge sword as the chunk rendering took twice the amount of time to render a chunk, fortunately I noticed that there were some potential optimizations as it sped up the rendering from 5 to 11 times faster.

I've also reduced memory usage by hundreds of megabytes depending on how many chunks are possibly visible, I only removed index an int variable and a vector3 position from the cube class as they can be easily generated if needed, as I mentioned a chunk has 4096 cubes and over 8000 chunks can be visible, the amount of memory needed for position and index is massive.

One down, thousands to go

So I finished creating the GUI based on Samuli's concept art and I can now start working on the gameplay... not.

I found a memory leak in the terrain generation that's resulting either in garbage collector not clearing or it's something I've done, I'm guessing both, so I have to trace it which means refactoring the terrain generation, although this prevents me from continuing the gameplay, but good news is that I can modify the terrain so I can create the actual size I was planning.

I was also planning in adding the cube unwrap which I've talked for so long now, but it dawned on me that the normals in the mesh would suffer, rendering a cube properly would need all 20 verticies otherwise I would need to rotate the angle of the normals and I have no idea what that would look like...

Rendering problems

So I had noticed in the beginning that there were problems with the GUI rendering transparent textures, the textures were missing either pixels or were rendered in the wrong order.

I used a transparent cutout shader to display the textures "properly", but there were problems of course, specifically because I used alpha testing to render the opaque pixels from the texture while writing to the depth buffer. In the next pass of the shader I turned depth buffering off and prevented any pixels from the depth buffer from being written again, after that I had turned alpha blending on and it wrote the transparent pixels. It all seemed fine and dandy but it did cut some of the more transparent pixels off.

I removed the first pass and now I only have the second pass in the shader (the shader is close to an unlit transparent but with vertex coloring) and it seems to render fine, expect for the sorting of the triangles in the mesh, which I had to add in to my mesh rendering. The only problem with it was with the rendering order with other gameobjects, they were either always shown on top of the GUI or the GUI renderers din't show over the others renderers even if they were closer to the camera. 

This was really weird and I looked everywhere for this problem, I though that the problem was in the mesh as the unity's cube mesh would render correctly with the same shader and this puzzled the hell out of me. Finally I found the answer and it was in the material, more precisely in the renderQueue member of the material. The renderQueue was set to 1 every time a texture is changed by script in the material (maybe by modifying other members in the material as well), the number in the renderQueue is the same tag as the shader's Queue tag which are:

1. Background, 1000
2. Geometry, 2000
3. AlphaTest, 2450
4. Transparent, 3000
5. Overlay, 4000

The renderQueue tag is set automatically if the material is modified by unity, but if it is modified by script it will set it to 1 which is nothing and it would result in a very chaotic rendering order, by setting the renderQueue to 3000 would result in a correct rendering order with other gameobjects.

Moving on

I finally finished the new GUI framework which took ages to complete, but now I can finally have an easier and less frustrating time creating the GUI for this project. I've also tried to make the framework as user-friendly as possible while being flexible and able to do complex GUI's.

I've also began creating the GUI for the game using the textures and concept art I received from Samuli, when it's done I can continue working on the gameplay for the game, specially the character's skills.

I'm also eager to change the chunk size to 16 * 16 * 16 from 16 * 16 * 8 as well as adding the cube unwrapping so I can reduce the vertex count.

Upgrading

I recently upgraded from unity 4 to 5.4 and to my surprise the game is running more smoothly specially around trees.

Why I only now upgraded to unity 5 was because I did not believe it would boost the game in anyway specially in how unity handles custom mesh data (I wish unity would take arrays as references or at least give an option so that I can give it an array with an integer to let it know of how much of the array I'm using so I don't have resize the array every time...).

I also began to reprogram the GUI for the game, because it was lacking in features and it was quite a pain to work with, why I don't use an existing GUI is because I know what I want from it and I can optimize it for that purpose and to make it work the way I want it to. You might think it's reinventing the wheel and you would be right but if people don't at least study or try to create something that has already been done they don't advance as a programmer, besides I've already done this once and I can create modding support and modding tools for it at the same time.

I've also been thinking if I should sell the GUI in the Asset Store, but I'm not sure if anybody will buy it and if it's worth all the hassle to get it into the Asset Store. I also been thinking if I should join Patreon and if I do then I'll just share it for free there...

More on the GUI, it will be fast, responsive, automatic scaling and relocation based on resolution, it will have buttons (with a delegate, mouse over, down and normal textures and a tooltip), uniform and sliced scaling, text (which is rendered from a bitmap), able to use a texture and render it in a circle with a varying central angle and to create a drop down menu with menu items hidden outside the menu even when scrolling, the GUI will be stored in an XML file and if the GUI was not saved it will be also serialized to the scene or prefab so no changes will be lost, also because it can be saved into an XML file changing the GUI while the game is running is possible and it's possible for the user to change the GUI, this of course can also be disabled, the GUI can be modified either via the scene or script.

Information for you, sir.

So I haven't told much about my game other than it having epic spells, "living" AI and things to build from items to cities and governing of cities, what I've yet to tell you is that as you, the player, can be mostly anything in the game and I'll try to make it interesting, of course being a peasant can get quite dull after some time but you can always move on from that from a peasant to a dictator having peasants.

What I'll try to achieve in this game is a living world where the npcs can do something big which can change a city by a lot, for example an npc finds a book of undead and decides if he should start reading it or not, if the npc begins reading the said book he will probably change into a necromancer and built a cult somewhere near the city and most likely attack the place after the npc has gained enough power. This might sound big and very scripted, sure there is "scripting" involved but I'll try to make it very random on how it will proceed based on the npc's personality.

There is also a concept of good and evil but its not based on what we declare what is good or evil but
based on the religion of the village, which can change over time, a village might find cannibals saints and people who help others evil.

Speaking of religion the gods of a religion might or might not exist at all depending on how many believes in a god of specific religion, by this way it is possible to make the people of a vast land believe or fear you so much that you might gain divine powers, what you can do with these divine powers I've yet to design but one possibility is creating a new race (which should be easy as everything is random generated), it would also be possible that npcs could disguise them as you or some other god and gain popularity in areas or possibly even overthrow you as their god.

Now you might think that doing all this is impossible or a project that will last for an infinity, you might be right but that's why it's called a seemingly impossible project. All of the things above need careful planning and designing before they can be programmed and I've already began doing all the planning and designing. Most of the world will be simulated in a very high level by using time as the controlling variable, everything important like kings and queens, leaders, necromancers, people you meet, people you love, people you work with are simulated in a low level so they actually do stuff in the game, the more you are involved with an npc the finer the npc is simulated.

Of course that's not all there is to the game as you can do all sort of things from crafting to skinning beasts, you will level up your skills as you do a specific task over and over, for an example if you poke enemies with your dagger enough you will be a master poker and a master in using daggers but you also gain better skill to weapons in the same category of one-handed weapons.

That is all for now, in a moment I'll forget everything and I'll have to read all of this before I can continue working on the game again, but hopefully at some point, at some, most likely a distant year I've finished this project (Seriously though I'll be working on this game every week and most likely everything here that I mentioned might be removed or changed, who knows).

Rivers, streams and more optimizations

I found a way to optimize the noise function I'm using by reversing how I calculate it and without calling it for all the cubes in the same X and Z coordinate in one chunk, but just changing the Y calculation in the end of the noise function. This allowed me to create the heightmap 100 (or something like that) times faster.

Why I began poking around the noise function was because I'm trying to add water to the game but
adding rivers and streams is very hard (at least making it fast enough), so incidentally looking for a way to add rivers via heightmap I found the optimization, although it didn't help me much in adding the rivers. I need to add some sort of flood fill algorithm to the noisemap to add rivers (streams are harder) and modify the values accordingly, though this is quite hard as overflow of the water can happen easily...

I also received new GUI graphics from Samuli which I have to add to the game, but before that I have to reprogram the GUI framework so I can modify the GUI more easily and add more features to it as well, I'll also add support for players to modify the GUI.

Here's also a webm of the trees that I got working in the game, I'm a little worried on how much objects I can actually let the player see before it fries their GPU.

View post on imgur.com

Data, so much data...

These past two weeks I've been fixing and creating the saving functionality and adding trees to the game, as I began to add trees to the game, I noticed how much data I have to store and process.

Currently I only store chunks that have been modified by anyone in the game and I'm looking at storing the noisemap, heightmap, treemap and biomemap, currently the problem is storing the treemap, it's easy to store if a certain location in a 2D plane has a tree but if I want to store multiple trees into a certain location, I have to store the height and the type of the tree, of course I could store this data in chunks like.. the chunks, but I could save quite a bit of space if I don't store the type of a tree but just the height but even then it's quite much.

I'm planning on creating a map of a size of 131072 * 131072 * 512 which is quite large (I don't plan on creating an infinite map because there is no point in that), the treemap for multiple trees in a certain spot would be a short type which is 16 bits which in turn is 16 trees upwards, this means that the treemap would be 274877906944 bits which is 34 gigabytes, sure I could compress this to make it smaller but I don't see the point because I plan to have the data ready to be used in the game and compressing and decompressing takes time (Because I plan to have multiple save slots I will compress the whole save to save space), but just as I wrote earlier I will be storing these things in chunks so the game won't eat the whole 34 gigabytes from beginning.

As I don't know how much of the world a player will actually discover I can't really estimate how much space a single game would take, so I'm just trying to plan ahead and try to minimize how much space a save will take.

The cube chunk is stored directly in bytes, int index, float x, float y, float z and then all of the 2048 cubes but only the cube type which is a short, so in total a chunk is 4112 bytes which is around 4 kilobytes.

If you're interested in what I've already done, check the older post to view the list of tasks.

Onwards

For now I've finished the basics of cube construction and removal so that the player's character
will move and do the certain tasks. I've also gained more GUI graphics, textures and trees from Samuli which I need to add.

Here are some of the things that I thought of that need to be either fixed or done.

- Fix the mouse raycast to work better, specially to be able to fire upwards. // Fixed and optimized.

- Fix the problem that the character floats when building or removing.

- Fix the movement of the character so it does not get stuck.

- Fix the pathfinding lag when building/removing cubes.

- Fix the issue when using actionpoints in pathfinding that the character is
not able to move to the edge of the map.

- Update the GUI with Samuli's new graphics.

- Refactor map generator and make it so that it can generate much bigger maps.

- Add a timer bar above the character when it's building or removing cube.

- Gain cubes when mining them and use cubes when building.

- Saving and loading of chunks.

- Saving chunk exists check to a file.

- Saving and loading of other map related things and characters.

- Add the trees Samuli made and so that they can be cut down.

- Create tree creator.

- Add iron veins.

- Add crafting (food, potions, equipments).

- Add equipment and items weapons/armor/potions/etc.

- Add graphics for items.

- Add the rest of inventory functionality (dragging items done).

- Add better way to build or remove cubes.

- Add a 3D model for the character.

- Add magic functionality.

- Finish the combat system, so you can dodge/block/etc when an enemy is attacking.

- Add weapon skills.

- Add enemies and neutrals to roam around.

- Make npcs fight each others.

- Add loot and equipment to npcs.

- Add foraging for food.

- Add "fluid simulation"

- Add basic survival gameplay.

- Program so that the character can be seen indoors, possibly by cutting everything above the character and do a sort of reverse frustum culling to the camera so it cuts everything between the character and camera.

- Try to find a way to view stuff inside a house or cave so they don't get cut out (possibly do a some sort of fluid simulation).

- Add interraction with npcs.

- Add water and plants.

- Add caves.

- Possibly recreate the GUI system.

- Add better functionality to the cube creator.

Work work

I've been currently working on the gameplay so I could present this project to other people and for myself so that I can see if this project is going to work or not, as well as making this an actual game rather than a tech demo.

What I've already done are rather simple things such as basic turn-based combat where the player can move a certain amount and strike the enemy if the player has enough actionpoints, the same thing is done for the AI, it moves closer to the player and attempts to kill the player, it's crude and not finished but it's still something.

Other than that I've been working on spell list and inventory, I've create an item creator from which I can in the future add items with descriptions and other things, this data is then stored in an XML file (I don't really like XML but as it is native to C# it was the quickest way) which a player can modify as well.

I only programmed the item creation so far as I have yet to figure out what I actually need to store into an item but for the moment these are the variables :

int id;
string name;
string description;
string iconSpriteName;
ItemTypes itemType;
int maxStack;
string cubeName;
int cubeIndex;

The ID is self explanatory as is name, description, icon sprite name, max stack and item type, but for the cube name it's actually the cubes ID from which I can get the cubeIndex, the reason why I use cube name as the ID is because as I started creating the cube editor I figured it was all I need, but it dawned on me, it would be easier just to use and actual integer ID like that for the item... So the cube index is gained at runtime with the help of each cube name.

What I've yet to create and finish are the inventory so that a player can pick up items, use and equip them, spells also need to be programmed as I have not yet touched them, I also need to add more tasks for the character so it can shovel/mine cubes and place cubes but this requires the inventory to function and lastly I need to add levels and other statistics for the players character which will increase as the players does something.

I don't know if that was all that I will be adding but I only really want to create the basic structure of the gameplay so I can expand it and modify it if does not work.

There are some things that worry me, like the angle of the projection from which the world is seen from because as I create a pretty big world can it be seen in all it's glory from that angle, is the player satisfied with it? Other thing that worries me is the gameplay if its fun to play but that is an universal problem with every game so I do try not to worry about it.

After I have finished or should I say satisfied with the basic gameplay I will continue on the world generation for a short while to adjust things and maybe add an optimization, the optimization is that I will unwrap every cube's textures into a single large texture so I can combine verticies together, currently all sides of a cube are rendered separately, so if I stack one cube's textures next to each other I can use them like this cube unwrap.

After that I'll move on to programming the first village, I will start by simulating the village in a very high level which would only mean generating resources and expanding the village only by calculating and then continue to a very low level at which I will simulate every characters needs and such.