TWID #27: 3D Scans in Unity

Exploring the use of 3D Scans in unity for interactive visuals.

Jul 10, 2023

This week I was using 3D Scans in unity to further improve my skills and to explore their potential for interactive visuals.

Preparing the assets

The exports from Scaniverse have a really high poly count, so the first step is to reduce it with the decimate modifier in blender. This seems to preserve the UVs which is essential for the textures.

Organising assets in unity

The import in unity is straight forward. You drop the model to your assets, create a new material containing the texture as a base map and then apply it to the model in your scene.

While doing this I realised the need for a naming pattern for my assets. Inspired by the way You Fujishiro is naming his elements I came up with this pattern:

Type_Scene_Name

Example:
MT_02_MaterialName

This helps me stay organised when exploring within different scenes and reference my assets quickly and reliably.

Playing with light

I wanted total darkness and a glowing ball that moves around and lights some part of the model.

To create total darkness you first set the background type to solid color and use black in the main cameras environment component. Then go to Window > Rendering > Lighting and under the environment tab set the light map intensity multiplier to 0 to get rid of all environment lights.

To create a light emitting ball you create a sphere with a Lit Material with emission set to white with higher HDR intensity. Then you place the point light at the same location and increase its brightness until you like it.

Attention.
When you place light inside a model make sure to set its material to transparent and blending mode to alpha. Otherwise you won’t see anything (a light inside a sphere only shines inside) even if it looks right in the editor.

Setting up the new input system with midi

When I first tried to use the new input system I was completely lost and even with some help didn't quite understand how it is working. So to improve that I took a short course on the unity learn platform and it really helped a lot.

The way I was using the input system was by adding a Player Input component to the Game Object and created a new Input Action Asset.

In there you can add custom actions and assign them different bindings. In this case I only used midi, but you can add other input devices too.

The input action behaviour is set to Send Messages, which shows a list of all possible messages it send to the game object. Those can be used in a script like this.

void OnFire()
{
  // Executes this code once when Button is pressed
  Debug.Log("OnFire");
}

void OnRotate(InputValue value)
{
  // Gets a Vector 2D with positive and negative values
  Vector2 inputValue = value.Get<Vector2>();
  Debug.Log("OnRotate: " + inputValue);
}

void OnScale(InputValue value)
{
  // With type midi note gets velocity even when pressed
  float inputValue = value.Get<float>();
  Debug.Log("OnScale: " + inputValue);
}

One thing that is quite helpful is the input debugger under Window > Analysis > Input Debugger which shows all the incoming midi messages.

Also keep in mind that you need to add a Midi Device Assigner Script to the Game Object in order for this to work properly.

Building for the iPad

I have the desire to build an app for the iPad to play visuals with by controlling it with a midi controller. I did not come to the point to try building it for that. But I noticed that I would need a keyboard and a mouse to use stage manager properly, if that is even possible in there. And I also would need an apple developer account to build it as a real app too use in there.

Maybe its too wild and not necessary, bit I still want to try it at some time.

Conclusion

I have learned a few things that got me further on my journey with unity. It is always challenging to use it, since it is just a huge piece of software. I have accepted that its takes time and practice to be able to use it really creatively.

But taking courses along the way helps to understand core concepts. It also opens up my mind to consider using gameplay mechanics for visuals and utilising them to bring some form of storytelling in the shows.

I still see huge potential in the software especially with lights, models and shaders. and all the other things that can be explored there.