Deep Learning and Computer Vision in R: A Practical Introduction (1.2)

BIBC2025 workshop - reticulate basics

Patrick Li

RSFAS, ANU

Content summary

• Overview of computer vision (CV)
• > reticulate basics
• Image classification
• Hyperparameter tuning
• CV model interpretation
• Object detection
• Image segmentation

R Package reticulate

Almost all modern deep learning frameworks and most computer vision tools are implemented in Python.

reticulate is an R package that lets you call Python code seamlessly from R.

• Converts common R and Python objects automatically for easy exchange
• Supports multiple Python versions and environments

Installation

Please run scripts/setup.R to set up reticulate and Python.

The script will:

1. Install or update the R packages required for this workshop, including reticulate
2. Install Miniconda, a Python environment manager
3. Create a new conda environment (similar to an R project, but with associated R packages)
4. Install the Python libraries needed for the workshop in the conda environment

If you have any installation issues, please let me know.

Use the conda environment

All the required packages and libraries are already installed in the ibsar-cv-workshop conda environment.

We can instruct reticulate to use this environment with the following code:

library(reticulate)
library(scrubwren)
use_condaenv("ibsar-cv-workshop")

To verify that reticulate is correctly linked to the environment, run:

py_config()

Common objects in Python

In Python, mutable objects can be changed after creation, while immutable objects cannot be modified once created.

x = [1, 2]
y = x
x[0] = 2  # Changing x also changes y because both refer to the same list

Immutable:

• Int: 1
• Float: 1.0
• Boolean: True
• Str: 'abc'
• Tuple: (1, 2, 3)

Mutable:

• List: [1, 2, 3]
• Dict: {'a': 1, 'b': 2}
• Numpy array: array([[1, 2, 3], [4, 5, 6]], dtype=int32)

Object conversions (R to Python)

Integer (be careful), float, string and logical

r_to_py(1L)

1

r_to_py(1)

1.0

r_to_py("abc")

'abc'

r_to_py(TRUE)

True

NULL, NaN and Inf

r_to_py(NULL)

None

r_to_py(NaN)

nan

r_to_py(Inf)

inf

Object conversions (R to Python)

NAs (depends on R internal)

r_to_py(NA)

True

r_to_py(NA_integer_)

-2147483648

r_to_py(NA_real_)

nan

r_to_py(NA_complex_)

(nan+nanj)

Vector

r_to_py(c(1, 2, 3))

[1.0, 2.0, 3.0]

r_to_py(list(1, 2, 3))

[1.0, 2.0, 3.0]

r_to_py(vector(mode = "list", 
               length = 3))

[None, None, None]

Object conversions (R to Python)

Named list

r_to_py(list(a = 1, b = 2))

{'a': 1.0, 'b': 2.0}

r_to_py(head(cars))

{'speed': [4.0, 4.0, 7.0, 7.0, 8.0, 9.0], 'dist': [2.0, 10.0, 4.0, 22.0, 16.0, 10.0]}

Array

r_to_py(matrix(1:4, ncol = 2))

array([[1, 3],
       [2, 4]], dtype=int32)

Function

(foo <- r_to_py(\(x) x + 1))

<function make_python_function.<locals>.python_function at 0x11f5ff6a0>
 signature: (*args, **kwargs)

foo(1)

2.0

Non-convertible R objects

r_to_py(y ~ x)

<capsule object "r_object" at 0x12c364630>

Object conversion (Python to R)

Python list will be simplified to atomic vector when possible.

py_to_r(py_builtins$list(
  list(1, 2, 3)
))

[1] 1 2 3

py_to_r(py_builtins$list(
  list(1, 2, list(3, 4))
))

[[1]]
[1] 1

[[2]]
[1] 2

[[3]]
[1] 3 4

Some Python objects are non-convertible.

py_builtins$set(c(1, 2, 3)) |>
  py_to_r() |>
  print() |>
  class()

{1.0, 2.0, 3.0}

[1] "python.builtin.set"    "python.builtin.object"

torch tensor

Import torch

torch <- import("torch", convert = FALSE)

Create torch tensor

• data type (torch$float32, torch$int32, …)
• storage type/device ("cpu", "cuda", "mps", …)
• torch$accelerator$current_accelerator()

x <- torch$tensor(matrix(1:4, ncol = 2),
                  dtype = torch$float32,
                  device = "cpu")
x

tensor([[1., 3.],
        [2., 4.]])

torch tensor operations

x + x

tensor([[2., 6.],
        [4., 8.]])

x * x

tensor([[ 1.,  9.],
        [ 4., 16.]])

x %*% x

tensor([[ 7., 15.],
        [10., 22.]])

x$mean()

tensor(2.5000)

Many other operations can be performed on tensors, check out the documentation.

torch tensor indexing

Python uses 0-based indexing, and torch indexing syntax is quite similar to that of R.

x[0]    # First element along the first dimension

x[0, 0] # First element of the first dimension, 
        # first element of the second dimension
        
x[, 0]  # First element along the second dimension

x[, -1] # Last element along the second dimension

x[0:2]  # First two elements along the first dimension

x[0:-1] # All elements along the first dimension except the last

x[0] <- torch$tensor(5:6) # Replace the first element along the first 
                          # dimension with (5, 6)

Convert torch tensor back to R array

Unfortunately, safely converting a torch tensor back to an R array can be quite complicated, especially when the tensor resides on the GPU or is part of a computation graph.

It’s generally best to keep it as a torch tensor throughout your workflow whenever possible.

Best practice:

x$detach()$cpu()$numpy()$copy() |>
  py_to_r()

     [,1] [,2]
[1,]    1    3
[2,]    2    4

Compute gradient for \mathbb{R}^n \to \mathbb{R}

Define y = \sum_{i=1}^n x_i^2 + x_i.

x <- torch$tensor(c(1, 2), 
                  dtype = torch$float32, 
                  requires_grad = TRUE)
y <- torch$sum(x^2 + x)
y

tensor(8., grad_fn=<SumBackward0>)

Compute \left(\frac{\partial y}{\partial x_1}, \dots , \frac{\partial y}{\partial x_n}\right), which is (2x_1 + 1, \dots, 2x_n + 1 )

torch$autograd$grad(y, x)

(tensor([3., 5.]),)

Exercises 1.2

1. Generate a random tensor x and another random tensor e, each of shape (50), using torch$rand(). Use torch$float32 as the data type and enable gradient tracking.

2. Compute y as: \boldsymbol{y} = 1 + 3 \boldsymbol{x} + \boldsymbol{e}

3. Compute y_hat as: \hat{\boldsymbol{y}} = 2 + 2 \boldsymbol{x}

4. Compute the loss as: \text{loss} = \frac{1}{50} \sum_{i=1}^{50} (y_i - \hat{y}_i)^2

5. Compute the gradient of loss with respect to x.