I. Introduction

This document outlines the multi-dimensional tensor and its properties.

Used notably in machine learning, tensors are explored as a general-purpose, mathematical data type capable of representing values in an infinite number of dimensions.

II. Definition

In general, a tensor can be one of two things:

An arbitrary value
A list (or array) of tensors

In the first definition, I specified that a tensor can be an arbitrary value. In programmatic language, this means that a tensor can represent any data type, from bits to strings. The second definition suggests the multi-dimensional property of the tensor.

III. Properties

These properties are applicable to tensors of all types, shapes, and sizes.

Type

Specifies the data type that the tensor encapsulates. An unsigned 32-bit integer tensor encapsulates unsigned 32-bit integers, a byte tensor encapsulates bytes, and so on.

Dimensions

Specifies an arbitrary number of dimensions. A zero-dimensional (0D) tensor is commonly referred to as a scalar, and aligns with the first definition provided above (an arbitrary value). Dimensions 1 and 2 represent vectors (or lists) and matrices, respectively. The list goes on to a cube, a list of cubes, and so on.

Shape

Specifies the size of each dimension, often in the form of an array. Want to represent the position of an object in 2-dimensional space? Use a 1-dimensional vector with the shape [2]. In this tensor, each element would represent a component of the position (X and Y, respectively).

IV. Construction

Tensors are recursive by nature. Below is a C# method for generic tensor construction.

private static Tensor CreateTensor(ulong[] shape, int offset, int count, Func<Scalar> generate)
{
    var dimensions = (ulong) (count - offset);

    // A zero-dimensional tensor is a scalar, so
    // generate a value.

    if (dimensions == 0)
    {
        return generate();
    }

    // The size of this tensor is the number of elements (sub-tensors)
    // that it contains.

    var size = shape[offset];
    var tensor = new Tensor[size];

    for (ulong i = 0; i < size; i++)
    {
        // Initialize each element of the tensor with an empty tensor
        // in the next dimension.

        tensor[i] = CreateTensor(shape, offset + 1, count, generate);
    }

    return tensor;
}

For the non-technical, at each dimension D, the size N of D is known from the shape specified, and so we create N * (D - 1) tensors. If the number of dimensions is zero at any point, we generate a value.

Short Story

In 2021, I received a call from a recruiter for a phone screening. I was asked how I would teach recursion to someone who had never heard of it before.

Unprepared, I replied "I would explain to them that it's like the telephone game you play where you pass a message along, and at each stage the message gets changed, and at the end we're left with some final message."

Now, I would explain that it's a matter of breaking a problem down into chunks until you get to the smallest possible problem, at which point the solution to that problem bubbles all the way up and solves the original problem.

V. Notes

a. Recursive Tensor Visualization

// Tensor.Empty([5, 6, 7])
// Dimensions: 3
//
// [[[0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0]],
//  [[0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0]],
//  [[0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0]],
//  [[0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0]],
//  [[0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0],
//   [0, 0, 0, 0, 0, 0, 0]]]

// Tensor.Empty([6, 7])
// Dimensions: 2
//
// [[0, 0, 0, 0, 0, 0, 0],
//  [0, 0, 0, 0, 0, 0, 0],
//  [0, 0, 0, 0, 0, 0, 0],
//  [0, 0, 0, 0, 0, 0, 0],
//  [0, 0, 0, 0, 0, 0, 0],
//  [0, 0, 0, 0, 0, 0, 0]]

// Tensor.Empty([7])
// Dimensions: 1
//
// [0, 0, 0, 0, 0, 0, 0]

// Tensor.Empty([])
// Dimensions: 0
//
// 0

Tensor, M. 1