Note: This is the 4th post of a 5 part series.

Previous Post : Adding Third-Party Libraries - Part 3

Fun fun fun fun fun fun fun

It’s so fun being able to tinker around with this kind of stuff so freely. It truly is :). Want to see what the next close encounter with an asteroid will look like? Let’s throw the one after that in too. Oh, and can we also see ʻOumuamua’s trajectory??

Sure :)..

Keep in mind these are all debug assets & debug lines. That video is from this series’s sample app that I’ll be posting, not our actual SPICE-empowered game. Does anyone have a good model of the James Webb Space Telescope in FBX format? The only one I have is in a Maya file and I can’t remember where I put my Maya. Let’s make a visualization of it’s path frpm Earth to L2.

I also made a video of all 1 million+ asteroids known to JPL popping in, in the order of their discovery. Very interesting to watch the swirls of new asteroids which follow the earth’s track as we discover new ones in the neighborhood. Oh, hell, just watch for yourself if that sounds interesting :).

Next week I think we’ll put some machinery to fly in and see if we can record SPICE data, then analyze it with JPLs tools outside of Unreal Engine.

The good stuff, at last!

We’re getting close, that’s for sure. In this part we’re going to work on the layer that sits in-between CSPICE and Unreal Engine.

Okay, now that we have a CSPICE module built and linked to our app, what can we do with it?? Ultimately, we want Blueprints that leverage SPICE.

Data Models

We will develop a Spice module which is an intermediary and translates data between UE and SPICE. Let’s flesh this module out. We already know what types go into the CSPICE Library from browsing the documentation. But what does the UE data look like?

Unreal Engine’s serialization system doesn’t serialize just anything. Prior to UE5, it wouldn’t even serialize a double. That’s why this series is built on UE5, so we have doubles. SPICE is alllllll doubles, all the time. UE won’t serialize structures, unless they’re implemented specifically for Unreal as a USTRUCT.

IMO, even for things like “Distance”, “Speed”, and “Mass” it is helpful to have specific containers. Same for “Time”. Let’s give them unique types. An entire project can quickly become a mess when everything is a ‘double’ or ‘double [3]’ etc. So, even though their contained types may just be doubles, etc, we’re going to create separate types for things that are physically different. We may duplicate some code or whatnot, but a little of that won’t sink our project… But when conceptual clarity is absent, complexity tends to build around it. A lack of conceptual clarity will sink our project much more readily than a little duplicated code, etc.

And units. Units can become a HUGE mess on big projects like this. BIG. HUGE. Specialized containers will keep everyone on the same page regarding units. “Is this distance in meters, or kilometers?" Look in the USTRUCT! The USTRUCT stores its data in a member variable, ‘double kilometers’. End of mystery.

Spaceflight is complex. We optimize heavily for conceptual clarity in the space flight domain. It will involve a few tradeoffs. They’re worth it. It minimizes high-level complexity in the important domain - space flight.

Floating Point Types

While Unreal Engine rolls with singles, SPICE deals in doubles. Everything inside SPICE boils down to a double or double array. Single precision floats fall apart on the order of a couple of kilometers. This was a limiting factor in game-engine map sizes for a long time. Doubles aren’t perfect, but they can takes us much farther and they’re what SPICE is built around.

Coordinate System Handedness

Other differences? Yup. Coordinate System Handedness. Unreal Engine uses a “left-handed” coordinate system while SPICE uses a “right hand” one. Both choices make sense in their domain due to historical precedents for each.

Let’s postpone, for now, a discussion of exactly how rocket scientists pick their reference frames, other than to say…. They use right-handed coordinate systems. You can think of theirs, conceptually, as.. the Y and Z axes are exactly the same as Unreal Engine. But, in Unreal Engine the X axis is the Left hand version (“in”/“forwards”), and in SPICE land it’s the Right Hand version (“out”/“back”).

So, we’re going to have a hard boundary here, where coordinates, directions, etc are on one side (the single precision, LHS, UE5 world) or the “other” side (the double precision, RHS, SPICE world). Any data that moves between these two worlds will be transformed to/from by our SPICE module that sits in between the game/app modules and the SPICE_Library module.

Th Spice layer is going to change handedness by swapping the X and Y axes. When I worked on Direct3D we called this a “swizzle”, and we will be doing a .yxz swizzle here.

We’re going to support interchanging only two types of spatial data between UE and SPICE - positions etc as cartesian vectors, and rotations as quaternions.

FVector USpiceTypes::Swizzle(const FSDimensionlessVector& value)
{
    // Swizzle the SPICE RHS Coordinate System to UE LHS System by swapping x, y...
    // And demote it to a float.
    return FVector((float)value.y, (float)value.x, (float)value.z);
}

FVector USpiceTypes::Conv_SDistanceVectorToVector(
    const FSDistanceVector& value
)
{
    // Any time we pass coordinate system data across the SPICE/UE boundary we have to
    // convert between the two coordinate systems... That's what Swizzle does.
    return Swizzle(value);
}

For rotations, we can account for the handedness and swizzling if we change a few of the signs as we swizzle around.

FQuat USpiceTypes::Conv_SQuaternionToQuat(
    const FSQuaternion& value
)
{
    return FQuat(-value.q[2], -value.q[1], -value.q[3], value.q[0]);
}

And if we constrain our spatial data to only those two types (no matrices, etc etc) and ensure goes through the appropriate swizzling layer - CSpice - then everything is going to be just fine despite the different coordinate systems.

Enums

And Enums. SPICE uses a lot of enums marshalled as strings, like "CN+S". ("CN+S", of course, means Converged Newtonian light time correction and stellar aberration correction). Instead of accepting strings for these functions in Blueprints let’s make enums. Infinitely more useable.

UENUM(BlueprintType)
enum class ES_UTCTimeFormat : uint8
{
    Calendar UMETA(DisplayName = "Calendar format, UTC"),
    DayOfYear UMETA(DisplayName = "Day-of-Year format, UTC"),
    JulianDate UMETA(DisplayName = "Julian Date format, UTC"),
    ISOCalendar UMETA(DisplayName = "ISO Calendar format, UTC"),
    ISODayOfYear UMETA(DisplayName = "ISO Day-of-Year format, UTC")
};

Straight forward enough. The UENUM macro ensures Unreal Engine’s reflection system can find it for serialization, etc. Note the BlueprintType inside the UENUM macro, we’ll want that so we can use this type in Blueprints.

Flags

In at least one case we’re going to want bit ‘flags’, instead of ordinal enums. That’s done like so:

UENUM(BlueprintType, meta = (Bitflags, UseEnumValuesAsMaskValuesInEditor = "true"))
enum class ES_ErrorDescriptionItems : uint8
{
    NONE = 0 UMETA(Hidden),
    Short = 1 << 0,
    Explain = 1 << 1,
    Long = 1 << 2,
    Traceback = 1 << 3,
    Default = 1 << 4 UMETA(DisplayName = "All (Default)")
};
ENUM_CLASS_FLAGS(ES_Items);

Note the UENUM meta Bitflags, etc and ENUM_CLASS_FLAGS macro.

We’re going to end up with many enums, most of them representing things that CSPICE marshals as strings.

Structs

Let’s look at one of the double quantities… “Time” variables.

Rocket Scientists have many possible representations of time. The default representation at JPL is et. This stands for Ephemeris Time. The main competitor, in the great time wars, is Barycentric Dynamical Time. There are others. Many others. Time is a complex topic. Anyways, CSPICE uses Ephemeris Time. This is a longwinded way of explaining what you’re about to see may seem overly pedantic and long winded. Time as et:

Time for some time code

USTRUCT(BlueprintType)
struct FSEphemerisTime
{
    GENERATED_BODY()

    // ephemeris seconds past J2000
    UPROPERTY(EditAnywhere, BlueprintReadWrite) double seconds;

    FSEphemerisTime()
    {
        seconds = 0.;
    }

    FSEphemerisTime(double _seconds)
    {
        seconds = _seconds;
    }

    /// <summary>Returns value in Seconds past J2000 Epoch</summary>
    /// <returns>Seconds</returns>
    inline double AsDouble() const { return seconds; }

    static SPICE_API const FSEphemerisTime J2000;
};

This USTRUCT is defined as Ephemeris time, relative to ‘seconds past the J2000 epoch’.

Unreal Engine requires USTRUCT names to begin with “F”. Why the S in “FSEphemerisTime”? It stands for SPICE. Anyways, what could have been a simple double is a full blown USTRUCT with a full blown UPROPERTY. Is it better to optimize at the low level and just have a bunch of doubles? Or does it make more sense to optimize at the high level, the conceptual level, and ensure everything remains conceptually clear? For me it is the latter. The codifying that this quantity is an “ephemeris time” in “seconds” relative to “12:00 Jan 1, 2000” is valuable.

The USTRUCT macro exposes the type’s structure to the UE reflection system so it can participate in serialization, blueprints, etc. BlueprintType ensures it’s exposed to the Blueprints system.

And, the structure’s data is stored in a UPROPERTY. This is marked to be editable in the UE editor. The user can clearly see the unit of measurement is ‘Seconds’.

Unreal engine requires a default (parameterless) constructor be available. As seen in the code above, it is.

And we just have a simple double constructor and AsDouble() accessor to add clarity to our code later. These are the entry points for converting between UE and SPICE types that the CSpice layer will call.

A static variable holds the EphemerisTime value for epoch J2000. (et=0.);

Simple.

Let’s also avoid “Everything’s a vector”. So, we’ll make bespoke containers for Positions, Velocities, etc. Matrices. Yup. Use your imagination. That slight sting you feel is your engineering sense, alarmed that we’re duplicating code since all these things boil down to the same fundamental containers at the low level - double [3]. Ignore it. We’re optimizing at a higher level.

Nested UE Types

One wrinkle, is that if we have scalar types for “distance”, “speed”, etc, we have a choice for “position” and “velocity”, which are 3-dimensional. Should we encode our multi dimensional containers using the single dimension base type, or use doubles?? Should the .x member of a 3D vector be double x or FSDistance x?

USTRUCT(BlueprintType)
struct FSDistanceVector
{
    GENERATED_BODY()

    UPROPERTY(EditAnywhere, BlueprintReadWrite) FSDistance x;
    UPROPERTY(EditAnywhere, BlueprintReadWrite) FSDistance y;
    UPROPERTY(EditAnywhere, BlueprintReadWrite) FSDistance z;

...

    static SPICE_API const FSDistanceVector Zero;
};

For positions, which we’re calling FSDistanceVector, child UPROPERTYs are declared as FSDistance types, as opposed to doubles. This is more useful in Blueprints, because if you “break” a SDistanceVector in a Blueprint you have 3 SDistance (x, y, z) elements. They represent distances, so you should end up with a SDistance, right?

We’re going to have a whole bunch of structures, just use your imagination :-D. These structures are our data model on the UE side.

Blueprint math operators

If we create a Blueprint Function Library, we can then generate some Blueprint operators.

UCLASS(BlueprintType, Blueprintable)
class SPICE_API USpiceTypes : public UBlueprintFunctionLibrary
{
    GENERATED_BODY()
public:

    /* Addition (A + B) */
    UFUNCTION(BlueprintPure, meta = (DisplayName = "time + period", CompactNodeTitle = "+", Keywords = "+ add plus"), Category = "Spice|Math|Time")
    static FSEphemerisTime Add_SEphemerisTimeSEphemerisPeriod(FSEphemerisTime A, FSEphemerisPeriod B);

This defines a “+” operator that can add a period to a time, to get a new time value. Blueprint Operators make math more friendly. When making an operator for blueprint, the C++ function name should be (operation)_(paramtype)(paramtype). And here, the parameter type does NOT contain the ‘F’ that prepends the C++ type (SEphemerisTime, not FSEphemerisTime). See the source code for the kismet math library for more examples.

Automatic type conversions

We can also add automatic type conversions, which make developing in Blueprints much more pleasant. For instance, we can generate an automatic conversion anyone tries to wire a value to a input in Blueprints like so:

    UFUNCTION(BlueprintPure,
        Category = "Spice | Api | Types",
        meta = (
            BlueprintAutocast,
            ToolTip = "Converts an angle to a double (radians)"
            ))
    static double Conv_SAngleToDouble(
        const FSAngle& value
    );

UE Blueprints will automatically add the conversion method when it sees a conversion is required and provided.

Note the C++ code above. For this to work the name here MUST be Conv_(input)To(output), where again the “F” is omitted. SDistance, not FSDistance. Also, note the BlueprintAutocast in the meta data.

Blueprint function nodes

Declaration

Okay, great, we have a whole bunch of data. Can we get to actually do some cool stuff already? Certainly!

We can make a Blueprint Function Library for SPICE. That’s cool, right? All methods in a function library are static, that way the user can just call “spkpos” without first constructing something that owns the method. Now, we start exposing SPICE functions to Blueprints in the Spice module intermediary.

    /// <summary>S/P Kernel, position</summary>
    /// <param name="targ">[in] Target body name</param>
    /// <param name="et">[in] Target epoch</param>
    /// <param name="ref">[in] Target reference frame</param>
    /// <param name="obs">[in] Observing body</param>
    /// <param name="pos">[out] Position of target</param>
    /// <param name="lt">[out] Light time</param>
    /// <returns></returns>
    UFUNCTION(BlueprintCallable,
        Category = "Spice | Api | Spk",
        meta = (
            ExpandEnumAsExecs = "ResultCode",
            ShortToolTip = "S/P Kernel, position",
            ToolTip = "Return the position of a target body relative to an observing body"
            ))
    static void spkpos(
        ES_ResultCode& ResultCode,
        FString& ErrorMessage,
        const FSEphemerisTime& et,
        FSDistanceVector& ptarg,
        FSEphemerisPeriod& lt,
        const FString& targ = TEXT("MOON"),
        const FString& obs = TEXT("EARTH BARYCENTER"),
        const FString& ref = TEXT("J2000"),
        ES_AberrationCorrectionWithNewtonians abcorr = ES_AberrationCorrectionWithNewtonians::None
    );

BOOM. There it is. Finally, a UFUNCTION

Something to note here…

The ExpandEnumAsExecs metadata tag creates multiple output execution pins, one pin for every value of the enumeration. Here, the enumeration type has two values, one for error and one for success. So, there’s two output pins. If the function succeeds, the node exits on the “Success” pin, if not it exits on the “Error” pin. Btw, SPICE gives you good error messages when it can’t do something. They’re returned to Blueprints in ‘ErrorMessage’.

Some Blueprint node inputs have sensible default values. In C++, these parameters must come last so they can be assigned default values. UE picks up on the defaults, and will initialize the node per their values. How does UE decide which parameters are Blueprint inputs versus outputs? We see ptarg and lt are not declared C++ const, so they show up as outputs. et was declared a C++ const, so it shows up in Blueprints as an input.

I’ve elected to retain all the SPICE names (spkpos, targ, pos, abcorr, etc). This makes it much easier to look up anything in the CSPICE documentation from NAIF. If I changed the name “spkpos” to “SPKernelPosition” someone searching for that wouldn’t find much. But searching for “spkpos” pops it right up.

Definitions

So, what does a UFUNCTION implementation look like? I’ve tried to make them rote, even when they’re kind of ridiculous.

1. Input
2. Output Defaults
3. CSPICE Invocation
4. Set Returns
5. Check for errors

void USpice::spklef(
    ES_ResultCode& ResultCode,
    FString& ErrorMessage,
    const FString& relativePath,
    int64& handle
)
{
    // Input
    auto gameDir = FPaths::ConvertRelativePathToFull(FPaths::ProjectDir());
    ConstSpiceChar* _filename = TCHAR_TO_ANSI(*(gameDir + relativePath));

    // Output
    SpiceInt        _handle = 0;

    // Invocation
    spklef_c(_filename, &_handle);

    // Return Value
    handle = int64(_handle);

    // Error handling
    ErrorCheck(ResultCode, ErrorMessage);
}

An example of the absurd:

void USpice::spkuef(
    int64 handle
)
{
    // Input
    SpiceInt _handle = handle;

    // Invocation
    spkuef_c(_handle);
}

Why not just this?

void USpice::spkuef(
    int64 handle
){
    spkuef_c((SpiceInt)handle);
}

Well, we do it the first way for clarity and consistency. Clarity and consistency go a long way towards keeping something useable.

In general, the Spice implementations are coded rotely:

// Tranform Inputs to CSPICE types
..

// Fill any return buffers with default values
..

// Invoke CSPICE
..

// Transform outputs to UE types and copy to output buffers
..

// Ask CSPICE if it signaled any errors and if so report them
..

Think about what the data should reallllly look like on the UE side

CSPICE has its own way of doing things. For example, see the documentation for ckcov_c. Long story made shorter, the CSPICE user has to do a bunch of complex stuff. We can automate all that and simplify it. The philosophy here is to pursue the user’s intent and make it useable, rather than simply mimic CSPICE and force the user to do it that way.

Similarly, a lot of CSPICE functions take string parameters, which boil down to stringified enumerations. So, as mentioned earlier, we make an enumeration.

Including the CSPICE headers

To use functionality in the CSPICE library we’ll need to include the CSPICE header files. However, they were authored in C, not C++. We’ll need to inform the compiler it’s looking at C and not C++. Wrapping the CSPICE headers in a extern "C" block does exactly that.

Another thing to note is unreal engine packs structs to a different alignment boundary than the default. That means our UE-aware code may interpret a CSPICE structure differently than the library did when it was build. You can go back to the original packing for the CSPICE headers and then reenable it using the PRAGMA_PUSH_PLATFORM_DEFAULT_PACKING/PRAGMA_POP_PLATFORM_DEFAULT_PACKING macros.

PRAGMA_PUSH_PLATFORM_DEFAULT_PACKING
extern "C"
{
#include "SpiceUsr.h"
}
PRAGMA_POP_PLATFORM_DEFAULT_PACKING

CSPICE shuts down the editor?!

One of the first issues with CSPICE was that the editor seemed to randomly shut down. This was because when CSPICE encounters an error its default action is exit(). That is highly annoying when the application is the UE editor.

The solution was a static constructor that changes the CSPICE error handling immediately after the module loads.

// Ensure CSpice doesn't try to exit the process...
// ...it's probably the UE editor, and that's just annoying.
class OnLoad
{
public:
    OnLoad()
    {
        SpiceChar szBuffer[] = "REPORT";
        erract_c("SET", sizeof(szBuffer), szBuffer);
    }
};
static OnLoad StaticInitializer;

Exceptions/Errors

Spice functions do not return error codes. As in the above snippet, the action-to-take-on-error is set by erract_c. The "REPORT" action means after we call spice and ask it if any errors occurred (failed_c()). And, if so - what it was (getmsg_c).

uint8 USpice::ErrorCheck(ES_ResultCode& ResultCode, FString& ErrorMessage)
{
    uint8 failed = failed_c();

    if (!failed)
    {
        ResultCode = ES_ResultCode::Success;
        ErrorMessage.Empty();
    }
    else
    {
        ResultCode = ES_ResultCode::Error;
        char szBuffer[LONG_MESSAGE_MAX_LENGTH];

        szBuffer[0] = '\0';
        getmsg_c("LONG", sizeof(szBuffer), szBuffer);

        if (!strnlen_s(szBuffer, sizeof(szBuffer)))
        {
            szBuffer[0] = '\0';
            getmsg_c("SHORT", sizeof(szBuffer), szBuffer);
        }

        ErrorMessage = szBuffer;

        UE_LOG(LogTemp, Warning, TEXT("USpice Runtime Error: %s"), *ErrorMessage);

        reset_c();
    }

    return failed;
}

There may be cases where these errors are perfectly fine. For instance, you may not have ephemeris data for the given object and at the given point in time and that may be okay.

Copy data files as part of build.

By default, Unreal Engine doesn’t recognize our SPICE kernel files as containing anything of value worth copying. One way to handle this is via Project Settings / Packaging / Additional Non-Asset Directories To copy. These directories will be copied into the correct location of the built game so that the CSPICE file readers can read the files.

Remember: There are no right or wrong solutions.

Just solutions optimized to different criteria.

As with anything, there’s no right or wrong solution here (assuming the solution works). I optimized for one set of criteria, and others would pick other criteria. Arguments about the “right” way to do something are best preceeded by agreeing on what “right” means in the first place.

That, in a nutshell, is how SPICE is exposed to UE and blueprints.

In the next post, let’s actually use this thing :-D. Finally, the truly good stuff!

Next Post : Adding Third-Party Libraries - Part 5