Tensors

Daniel Falbel

2019-02-04

In this article we will describe the Tensors API implemented by torch. A tensor is a multi-dimensional matrix containing elements of a single data type.

Torch defines eight CPU tensor types and eight GPU tensor types:

Data type	dtype	CPU tensor	CUDA tensor
32-bit floating point	float32 or float	FloatTensor	CUDAFloatTensor
64-bit floating point	float64 or double	DoubleTensor	CUDADoubleTensor
16-bit floating point	float16 or half	HalfTensor	CUDAHalfTensor
8-bit integer (unsigned)	uint8	ByteTensor	CUDAByteTensor
8-bit integer (signed)	int8	CharTensor	CUDACharTensor
16-bit integer (signed)	int16 or short	ShortTensor	CUDAShortTensor
32-bit integer (signed)	int32 or int	IntTensor	CUDAIntTensor
64-bit integer (signed)	int64 or long	LongTensor	CUDALongTensor

Tensor Creation

From R

We can create new tensors from R objects using the tensor function.

When creating tensors from R vector types we will use the following convertion table. Character tensors are not implemented yet.

R Type	dtype	Torch Type
Logical	uint8	ByteTensor
Integer	int	IntTensor
Double	float	FloatTensor
Character	int8	CharTensor

Note By default we cast doubles to floats because we can achieve much better performance on GPU.

# Vectors
x <- runif(5)
x
#> [1] 0.1534171 0.6500084 0.5318366 0.8589623 0.4184815
x_ten <- tensor(x)
x_ten
#> tensor 
#>  0.1534
#>  0.6500
#>  0.5318
#>  0.8590
#>  0.4185
#> [ Variable[CPUFloatType]{5} ]

# Matrixes
x <- matrix(runif(4), ncol = 2)
x
#>           [,1]      [,2]
#> [1,] 0.5849283 0.7149474
#> [2,] 0.5898070 0.8677066
x_ten <- tensor(x)
x_ten
#> tensor 
#>  0.5849  0.7149
#>  0.5898  0.8677
#> [ Variable[CPUFloatType]{2,2} ]

# Arrays
x <- array(runif(8), dim = c(2,2,2))
x
#> , , 1
#> 
#>           [,1]      [,2]
#> [1,] 0.4055101 0.3402357
#> [2,] 0.2818084 0.7347478
#> 
#> , , 2
#> 
#>           [,1]      [,2]
#> [1,] 0.8960093 0.6873888
#> [2,] 0.4709765 0.3205751
x_ten <- tensor(x)
x_ten
#> tensor 
#> (1,.,.) = 
#>   0.4055  0.8960
#>   0.3402  0.6874
#> 
#> (2,.,.) = 
#>   0.2818  0.4710
#>   0.7347  0.3206
#> [ Variable[CPUFloatType]{2,2,2} ]

Using a Factory Function

The following factory functions are available at the time of this writing:

• tch_arange: Returns a tensor with a sequence of integers,
• tch_empty: Returns a tensor with uninitialized values,
• tch_eye: Returns an identity matrix,
• tch_full: Returns a tensor filled with a single value,
• tch_linspace: Returns a tensor with values linearly spaced in some interval,
• tch_logspace: Returns a tensor with values logarithmically spaced in some interval,
• tch_ones: Returns a tensor filled with all ones,
• tch_rand: Returns a tensor filled with values drawn from a uniform distribution on [0, 1).
• tch_randint: Returns a tensor with integers randomly drawn from an interval,
• tch_randn: Returns a tensor filled with values drawn from a unit normal distribution,
• tch_randperm: Returns a tensor filled with a random permutation of integers in some interval,
• tch_zeros: Returns a tensor filled with all zeros.

For example we can create a tensor with:

x <- tch_randn(c(2,2))
x
#> tensor 
#> -0.5173 -0.3280
#> -0.0775 -0.0068
#> [ Variable[CPUFloatType]{2,2} ]

Configuring Properties of the Tensor

There are 4 construction axis for a tensor, they are:

• The dtype, which controls the data type of the elements stored in the tensor,
• The layout, which is either strided (dense) or sparse,
• The device, which represents a compute device on which a tensor is stored (like a CPU or CUDA GPU),
• The requires_grad boolean to enable or disable gradient recording for a tensor,

If you are used to PyTorch in Python, these axes will sound very familiar. The allowed values for these axes at the moment are:

• For dtype: uint8, int or int32, float or float32 and double or float64,
• For layout: strided or dense,
• For device: cpu, (no cuda support yet),
• For requires_grad: either TRUE or FALSE.