So I am currently working as a HiWi (assistant for all kinds of stuff at Uni) and I was tasked with building a solution for a Augmented Audio experience in Unity.

My boss wanted this for a seminar and I had to create the Framework for the students. The idea for the seminar was the following:

Build a mobile game which utilizes GPS tracking and heading to create a 3D audio “Soundscape” which you can walk through. It’s basically like any other GPS based game (e.g. Pokemon Go or stuff like that) but only (or mostly) works with audio.

So I had a look around online and came across two libraries to achieve GPS tracking in Unity: MapNav and Mapbox.

MapNav was scrapped right away, since my boss had already tried it and was not too pleased with it’s features. So Mapbox it is!

After installing the SDK and poking around at some interfaces I started planning my approach.

I wanted to have very distinct features:

  • Register GameObjects to coordinates in-editor (no copy pasting coordinates from maps or stuff like that!)
  • Real-Time updating of objects if/when the map object moves
  • Stationary objects and moving objects

With that and some semblance of an architecture planned out I started working.

First I needed a class for storing and manipulating GeoData (Lat/Lon values). I came up with this GeoPosition class, which handles all that stuff, and is editable in-editor. Note that I used Deadcows’ MyBox package to add Buttons to the inspector. I cannot recommend this package enough! It’s absolutely perfect to create highly flexible and easy to use frameworks for your designers!

[Serializable]
    public class GeoPosition
    {
        [Tooltip(
            "Update frequency of position polling. Update every n-th frame. 1 is every frame, 60 is every 60th frame.")]
        [Range(1, 60)]
        public int positionUpdateFrequency = 1;

        [Tooltip("Should the object have a specified altitude?")]
        public bool useYOffset = false;

        [Tooltip("If useMeterConversion is activated the yOffsets unit is meters, otherwise its unity units.")]
        public bool useMeterConversion = false;

        [Tooltip("The actual y position of the object in m or unity units depending on useMeterConversion.")]
        public float yOffset = 0;

        [Tooltip("X is LAT, Y is LON")]public Vector2d geoVector;

        [HideInInspector] public float worldRelativeScale;

        [HideInInspector] public MonoBehaviour parentReference;

        public Vector3 GetUnityWorldSpaceCoordinates(AbstractMap map)
        {
            UpdateWorldRelativeScale(map);
            var worldSpaceCoordinates = map.GeoToWorldPosition(geoVector, false);
            if (useYOffset)
            {
                worldSpaceCoordinates.y = yOffset;
            }

            return worldSpaceCoordinates;
        }

        public void UpdateWorldRelativeScale(AbstractMap map)
        {
            worldRelativeScale = map.WorldRelativeScale;
        }

        public void SetGeoVectorFromRaycast(Vector3 position, AbstractMap map, LayerMask layerMask)
        {
            var ray = new Ray(position, Vector3.down);
            if (Physics.Raycast(ray, out var hitInfo, Mathf.Infinity, layerMask))
            {
                geoVector = map.WorldToGeoPosition(hitInfo.point);
            }
            else
            {
                Debug.LogException(new NullReferenceException("Raycast did not hit the map. Did you turn on map preview?"),parentReference);
            }
        }

        public void SetYOffsetFromRaycast(AbstractMap map, Vector3 position, LayerMask layerMask)
        {
            UpdateWorldRelativeScale(map);
            // using raycast because of possible y-non-zero maps/ terrain etc.
            var ray = new Ray(position, Vector3.down);
            if (Physics.Raycast(ray, out var hitInfo, Mathf.Infinity, layerMask))
            {
                var worldSpaceDistance = Vector3.Distance(position, hitInfo.point);
                if (useMeterConversion)
                {
                    yOffset = worldSpaceDistance * worldRelativeScale;
                }
                else
                {
                    yOffset = worldSpaceDistance;
                }
            }
            else
            {
                Debug.LogException(new NullReferenceException("Could not find map below. Is map preview turned on?"),parentReference);
            }
        }
    }

Basically this class was ran by any MonoBehaviour that needed to be positioned on a map. The positioning itself was done with the SetGeoVectorFromRaycast and SetYOffsetFromRaycast methods. I used the hitpoint of a Raycast downwards to calculate the corresponding LAT/LON values for this particular point. Since we were working in 3D space we also needed a way to control the height of an object above the ground. I used the same method with the Raycast to calculate the distance of the object from the map itself. With that we could place objects above one another and on different heights in general.

Stationary Placeable

These methods were driven from a BasePlaceable which in turn was derived from to create different behaviours.

public class BasePlaceable : MonoBehaviour
    {
        [Foldout("Base Placeable Settings", true)]
        [Header("Component References.")]
        [Tooltip(
            " You can assign this to a specific map. If you don't, it will find it's reference using the MapReferencer object")]
        public AbstractMap map;

        [Tooltip("Specify the Layer the map is added to.")]
        public LayerMask mapLayer;

        public GeoPosition geoPosition;

        protected int PositionUpdateCount;

        [HideInInspector] public bool initialized = false;

        protected virtual IEnumerator Start()
        {
            geoPosition.parentReference = this;
            if (map == null)
            {
                if (MapReferencer.Instance == null)
                {
                    throw new NullReferenceException(
                        "No MapReferencer object present in the scene. Please use the provided Map prefab or add a MapReferencer manually.");
                }

                map = MapReferencer.Instance.GetMapReference();
                // map = MapReferencer.Instance.mapObject.GetComponent<AbstractMap>();
            }

            if (string.IsNullOrEmpty(geoPosition.geoVector.ToString()))
            {
                throw new NullReferenceException("No geocode provided. Please assign a geocode to this object.");
            }

            yield return null;
        }

        #region Positioning Editor

        [ButtonMethod]
        public virtual void RegisterCoordinates()
        {
            map = MapReferencer.Instance.GetMapReference();
            geoPosition.parentReference = this;

            geoPosition.SetGeoVectorFromRaycast(transform.position, map, mapLayer);
#if UNITY_EDITOR
            if (!Application.isEditor) return;
            Undo.RecordObject(this, "Set Value");
            EditorUtility.SetDirty(this);
#endif
        }

        [ButtonMethod]
        protected virtual void RegisterYOffset()
        {
            map = MapReferencer.Instance.GetMapReference();

            geoPosition.SetYOffsetFromRaycast(map, transform.position, mapLayer);
#if UNITY_EDITOR
            if (!Application.isEditor) return;
            Undo.RecordObject(this, "Set Value");
            EditorUtility.SetDirty(this);
#endif
        }

        [ButtonMethod]
        public virtual void RelocateObjectToUnityCoordinates()
        {
            map = MapReferencer.Instance.GetMapReference();
            transform.position = geoPosition.GetUnityWorldSpaceCoordinates(map);
#if UNITY_EDITOR
            if (!Application.isEditor) return;
            Undo.RecordObject(this, "Set Value");
            EditorUtility.SetDirty(this);
#endif
        }

        #endregion

        #region Editor Funcs

        protected virtual void OnDrawGizmos()
        {
            Gizmos.color = Color.red;
            var position = transform.position;
            Gizmos.DrawLine(position, new Vector3(position.x, position.y - 20, position.z));
        }

        #endregion
    }

We used this particular base class to implement a StationaryPlaceable, which was used to create objects for AudioSources, Waypoints for MovingPlaceables, points for MeshGeneration to create placeable Trigger Colliders and much, much more!

For those of you wondering, a story built with this framework looks something like this: Mapbox Unity Screenshot

With this framework in hand we started building a pretty big project: A thriller story in which you need to assist Prof. Rabenknarr in saving the world from the apocalypse. This app will soon be available on Android. I’ll make a post about it when it’s done!

Thanks for reading! 🌍