Spatial Alignment between Two Hololens 2 Headsets using ARFoundation/Azure Spatial Anchors

Question

I'm working through this tutorial: https://mtaulty.com/2019/07/18/simple-shared-holograms-with-photon-networking-part-1/ with the hope of reproducing the shared coordinate system between two Hololens 2 headsets. I'm using Unity 2020, PUN2, ARFoundation and MRTK.

Because the tutorial is using WorldAnchors (WSA platform), which is a bit old, I'm trying to modify it to use ARFoundation. So far, the code I have as a result, seems to properly have the two headsets communicating via PUN2, but the blue cube as shown in the tutorial does not align between the headsets. The cube simply seems referenced to each headsets initial startup frame of reference. Below is the code. I've kept everything as one-to-one with the tutorial as possible, except where I felt I needed to swap WorldAnchors for ARAnchors and also where I swapped in a SpatialAnchorManager class to handle the Azure Spatial Service session since I found the tutorial's StartSession function didn't seem to work properly. Both AzureSpatialAnchorService.cs and PhotonScript.cs are attached to a root game object in the scene. Picture of the scene attached. Based on the debug logs I'm able to tell that the first headset is creating and saving an anchor to Azure and the second headset is able to find that same anchor. But I apparently am not performing a necessary transformation between headsets?

Can anyone suggest what I'm doing wrong and/or what specific edits that need to be made to get spatial alignment between headsets?

Thanks!

AzureSpatialAnchorService.cs:

using Microsoft.Azure.SpatialAnchors.Unity;
using Microsoft.MixedReality.Toolkit.Utilities;
using System;
using System.Threading.Tasks;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.WSA;

namespace AzureSpatialAnchors
{
    [RequireComponent(typeof(SpatialAnchorManager))]
    public class AzureSpatialAnchorService : MonoBehaviour
    {
        [Serializable]
        public class AzureSpatialAnchorServiceProfile
        {
            [SerializeField]
            [Tooltip("The account id from the Azure portal for the Azure Spatial Anchors service")]
            string azureAccountId;
            public string AzureAccountId => this.azureAccountId;

            [SerializeField]
            [Tooltip("The access key from the Azure portal for the Azure Spatial Anchors service (for Key authentication)")]
            string azureServiceKey;
            public string AzureServiceKey => this.azureServiceKey;
        }

        [SerializeField]
        [Tooltip("The configuration for the Azure Spatial Anchors Service")]
        AzureSpatialAnchorServiceProfile profile = new AzureSpatialAnchorServiceProfile();
        public AzureSpatialAnchorServiceProfile Profile => this.profile;

        TaskCompletionSource<CloudSpatialAnchor> taskWaitForAnchorLocation;

        //CloudSpatialAnchorSession cloudSpatialAnchorSession;

        private SpatialAnchorManager _spatialAnchorManager = null;

        public AzureSpatialAnchorService()
        {
        }
        public async Task<string> CreateAnchorOnObjectAsync(GameObject gameObjectForAnchor)
        {
            string anchorId = string.Empty;
            try
            {
                await this.StartSession();

                Debug.Log("Started Session");

                //Add and configure ASA components
                CloudNativeAnchor cloudNativeAnchor = gameObjectForAnchor.AddComponent<CloudNativeAnchor>();
                await cloudNativeAnchor.NativeToCloud();
                Debug.Log("After NativeToCloud");
                CloudSpatialAnchor cloudSpatialAnchor = cloudNativeAnchor.CloudAnchor;
                cloudSpatialAnchor.Expiration = DateTimeOffset.Now.AddDays(3);

                // As per previous comment.
                //Collect Environment Data
                while (!_spatialAnchorManager.IsReadyForCreate)
                {
                    float createProgress = _spatialAnchorManager.SessionStatus.RecommendedForCreateProgress;
                    Debug.Log($"ASA - Move your device to capture more environment data: {createProgress:0%}");
                }

                Debug.Log($"ASA - Saving room cloud anchor... ");
                await _spatialAnchorManager.CreateAnchorAsync(cloudSpatialAnchor);

                anchorId = cloudSpatialAnchor?.Identifier;

                bool saveSucceeded = cloudSpatialAnchor != null;
                if (!saveSucceeded)
                {
                    Debug.LogError("ASA - Failed to save, but no exception was thrown.");
                    return anchorId;
                }

                anchorId = cloudSpatialAnchor.Identifier;
                Debug.Log($"ASA - Saved room cloud anchor with ID: {anchorId}");
            }
            catch (Exception exception) // TODO: reasonable exceptions here.
            {
                Debug.Log("ASA - Failed to save room anchor: " + exception.ToString());
                Debug.LogException(exception);
            }
            return (anchorId);
        }
        public async Task<bool> PopulateAnchorOnObjectAsync(string anchorId, GameObject gameObjectForAnchor)
        {
            bool anchorLocated = false;

            try
            {
                await this.StartSession();

                this.taskWaitForAnchorLocation = new TaskCompletionSource<CloudSpatialAnchor>();

                var watcher = _spatialAnchorManager.Session.CreateWatcher(
                    new AnchorLocateCriteria()
                    {
                        Identifiers = new string[] { anchorId },
                        BypassCache = true,
                        Strategy = LocateStrategy.AnyStrategy,
                        RequestedCategories = AnchorDataCategory.Spatial
                    }
                );

                var cloudAnchor = await this.taskWaitForAnchorLocation.Task;

                anchorLocated = cloudAnchor != null;

                if (anchorLocated)
                {
                    Debug.Log("Anchor located");
                    gameObjectForAnchor.AddComponent<CloudNativeAnchor>().CloudToNative(cloudAnchor);
                    Debug.Log("Attached Local Anchor");
                }
                watcher.Stop();
            }
            catch (Exception ex) // TODO: reasonable exceptions here.
            {
                Debug.Log($"Caught {ex.Message}");
            }
            return (anchorLocated);
        }
        /// <summary>
        /// Start the Azure Spatial Anchor Service session
        /// This must be called before calling create, populate or delete methods.
        /// </summary>
        public async Task<bool> StartSession()
        {
            //if (this.cloudSpatialAnchorSession == null)
            //{
            //    Debug.Assert(this.cloudSpatialAnchorSession == null);

            //    this.ThrowOnBadAuthConfiguration();
            //    // setup the session
            //    this.cloudSpatialAnchorSession = new CloudSpatialAnchorSession();
            //    // set the Azure configuration parameters
            //    this.cloudSpatialAnchorSession.Configuration.AccountId = this.Profile.AzureAccountId;
            //    this.cloudSpatialAnchorSession.Configuration.AccountKey = this.Profile.AzureServiceKey;
            //    // register event handlers
            //    this.cloudSpatialAnchorSession.Error += this.OnCloudSessionError;
            //    this.cloudSpatialAnchorSession.AnchorLocated += OnAnchorLocated;
            //    this.cloudSpatialAnchorSession.LocateAnchorsCompleted += OnLocateAnchorsCompleted;

            //    // start the session
            //    this.cloudSpatialAnchorSession.Start();
            //}

            _spatialAnchorManager = GetComponent<SpatialAnchorManager>();
            _spatialAnchorManager.LogDebug += (sender, args) => Debug.Log($"ASA - Debug: {args.Message}");
            _spatialAnchorManager.Error += (sender, args) => Debug.LogError($"ASA - Error: {args.ErrorMessage}");
            _spatialAnchorManager.AnchorLocated += OnAnchorLocated;
            //_spatialAnchorManager.LocateAnchorsCompleted += OnLocateAnchorsCompleted;
            await _spatialAnchorManager.StartSessionAsync();
            return true;
        }
        /// <summary>
        /// Stop the Azure Spatial Anchor Service session
        /// </summary>
        //public void StopSession()
        //{
        //    if (this.cloudSpatialAnchorSession != null)
        //    {
        //        // stop session
        //        this.cloudSpatialAnchorSession.Stop();
        //        // clear event handlers
        //        this.cloudSpatialAnchorSession.Error -= this.OnCloudSessionError;
        //        this.cloudSpatialAnchorSession.AnchorLocated -= OnAnchorLocated;
        //        this.cloudSpatialAnchorSession.LocateAnchorsCompleted -= OnLocateAnchorsCompleted;
        //        // cleanup
        //        this.cloudSpatialAnchorSession.Dispose();
        //        this.cloudSpatialAnchorSession = null;
        //    }
        //}
        void OnLocateAnchorsCompleted(object sender, LocateAnchorsCompletedEventArgs args)
        {
            Debug.Log("On Locate Anchors Completed");
            Debug.Assert(this.taskWaitForAnchorLocation != null);

            if (!this.taskWaitForAnchorLocation.Task.IsCompleted)
            {
                this.taskWaitForAnchorLocation.TrySetResult(null);
            }
        }
        void OnAnchorLocated(object sender, AnchorLocatedEventArgs args)
        {
            Debug.Log($"On Anchor Located, status is {args.Status} anchor is {args.Anchor?.Identifier}, pointer is {args.Anchor?.LocalAnchor}");
            Debug.Assert(this.taskWaitForAnchorLocation != null);

            this.taskWaitForAnchorLocation.SetResult(args.Anchor);
        }
        void OnCloudSessionError(object sender, SessionErrorEventArgs args)
        {
            Debug.Log($"On Cloud Session Error: {args.ErrorMessage}");
        }
        void ThrowOnBadAuthConfiguration()
        {
            if (string.IsNullOrEmpty(this.Profile.AzureAccountId) ||
                string.IsNullOrEmpty(this.Profile.AzureServiceKey))
            {
                throw new ArgumentNullException("Missing required configuration to connect to service");
            }
        }
    }
}

PhotonScript.cs:

using System;
using System.Threading.Tasks;
using AzureSpatialAnchors;
using ExitGames.Client.Photon;
using Photon.Pun;
using Photon.Realtime;

public class PhotonScript : MonoBehaviourPunCallbacks
{
    enum RoomStatus
    {
        None,
        CreatedRoom,
        JoinedRoom,
        JoinedRoomDownloadedAnchor
    }

    public int emptyRoomTimeToLiveSeconds = 120;

    RoomStatus roomStatus = RoomStatus.None;

    void Start()
    {
        PhotonNetwork.ConnectUsingSettings();
    }
    public override void OnConnectedToMaster()
    {
        base.OnConnectedToMaster();

        var roomOptions = new RoomOptions();
        roomOptions.EmptyRoomTtl = this.emptyRoomTimeToLiveSeconds * 1000;
        PhotonNetwork.JoinOrCreateRoom(ROOM_NAME, roomOptions, null);
    }
    public async override void OnJoinedRoom()
    {
        base.OnJoinedRoom();

        // Note that the creator of the room also joins the room...
        if (this.roomStatus == RoomStatus.None)
        {
            this.roomStatus = RoomStatus.JoinedRoom;
        }
        await this.PopulateAnchorAsync();
    }
    public async override void OnCreatedRoom()
    {
        base.OnCreatedRoom();
        this.roomStatus = RoomStatus.CreatedRoom;
        await this.CreateAnchorAsync();
    }
    async Task CreateAnchorAsync()
    {
        // If we created the room then we will attempt to create an anchor for the parent
        // of the cubes that we are creating.
        var anchorService = this.GetComponent<AzureSpatialAnchorService>();

        var anchorId = await anchorService.CreateAnchorOnObjectAsync(this.gameObject);

        // Put this ID into a custom property so that other devices joining the
        // room can get hold of it.
#if UNITY_2020
        PhotonNetwork.CurrentRoom.SetCustomProperties(
            new Hashtable()
            {
                { ANCHOR_ID_CUSTOM_PROPERTY, anchorId }
            }
        );
#endif
    }
    async Task PopulateAnchorAsync()
    {
        if (this.roomStatus == RoomStatus.JoinedRoom)
        {
            object keyValue = null;

#if UNITY_2020
            // First time around, this property may not be here so we see if is there.
            if (PhotonNetwork.CurrentRoom.CustomProperties.TryGetValue(
                ANCHOR_ID_CUSTOM_PROPERTY, out keyValue))
            {
                // If the anchorId property is present then we will try and get the
                // anchor but only once so change the status.
                this.roomStatus = RoomStatus.JoinedRoomDownloadedAnchor;

                // If we didn't create the room then we want to try and get the anchor
                // from the cloud and apply it.
                var anchorService = this.GetComponent<AzureSpatialAnchorService>();

                await anchorService.PopulateAnchorOnObjectAsync(
                    (string)keyValue, this.gameObject);
            }
#endif
        }
    }
    public async override void OnRoomPropertiesUpdate(Hashtable propertiesThatChanged)
    {
        base.OnRoomPropertiesUpdate(propertiesThatChanged);

        await this.PopulateAnchorAsync();
    }
    static readonly string ANCHOR_ID_CUSTOM_PROPERTY = "anchorId";
    static readonly string ROOM_NAME = "HardCodedRoomName";
}

Answer 1

From reviewing the code and scenario, I am reading that this is in same local area. So, the ASA service will have the shared anchor as detailed in docs:

https://learn.microsoft.com/en-us/windows/mixed-reality/design/shared-experiences-in-mixed-reality

"Shared static holograms (no interactions) Leverage Azure Spatial Anchors in your app. Enabling and sharing spatial anchors across devices allows you to create an application where users see holograms in the same place at the same time. Additional syncing across devices is needed to enable users to interact with holograms and see movements or state updates of holograms."

The tutorial that Microsoft docs point to is here if it helps comparison to the other sample: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unity/tutorials/mr-learning-sharing-01