Posit AI Blog: torch 0.11.0

torch v0.11.0 is now on CRAN! This weblog publish highlights among the modifications included
on this launch. But you may all the time discover the complete changelog
on the torch web site.

Improved loading of state dicts

For a very long time it has been potential to make use of torch from R to load state dicts (i.e. 
mannequin weights) educated with PyTorch utilizing the load_state_dict() operate.
However, it was widespread to get the error:

Error in cpp_load_state_dict(path) :  isGenericDict() INTERNAL ASSERT FAILED at

This occurred as a result of when saving the state_dict from Python, it wasn’t actually
a dictionary, however an ordered dictionary. Weights in PyTorch are serialized as Pickle information – a Python-specific format much like our RDS. To load them in C++, with out a Python runtime,
LibTorch implements a pickle reader that’s in a position to learn solely a subset of the
file format, and this subset didn’t embrace ordered dicts.

This launch provides help for studying the ordered dictionaries, so that you received’t see
this error any longer.

Besides that, studying theses information requires half of the height reminiscence utilization, and in
consequence additionally is far sooner. Here are the timings for studying a 3B parameter
mannequin (StableLM-3B) with v0.10.0:

system.time({
  x <- torch::load_state_dict("~/Downloads/pytorch_model-00001-of-00002.bin")
  y <- torch::load_state_dict("~/Downloads/pytorch_model-00002-of-00002.bin")
})

   consumer  system elapsed 
662.300  26.859 713.484 

and with v0.11.0

   consumer  system elapsed 
  0.022   3.016   4.016 

Meaning that we went from minutes to only a few seconds.

Using JIT operations

One of the commonest methods of extending LibTorch/PyTorch is by implementing JIT
operations. This permits builders to put in writing customized, optimized code in C++ and
use it straight in PyTorch, with full help for JIT tracing and scripting.
See our ‘Torch outside the box’
weblog publish if you wish to be taught extra about it.

Using JIT operators in R used to require bundle builders to implement C++/Rcpp
for every operator in the event that they needed to have the ability to name them from R straight.
This launch added help for calling JIT operators with out requiring authors to
implement the wrappers.

The solely seen change is that we now have a brand new image within the torch namespace, referred to as
jit_ops. Let’s load torchvisionlib, a torch extension that registers many various
JIT operations. Just loading the bundle with library(torchvisionlib) will make
its operators accessible for torch to make use of – it is because the mechanism that registers
the operators acts when the bundle DLL (or shared library) is loaded.

For occasion, let’s use the read_file operator that effectively reads a file
right into a uncooked (bytes) torch tensor.

library(torchvisionlib)
torch::jit_ops$picture$read_file("img.png")

torch_tensor
 137
  80
  78
  71
 ...
   0
   0
 103
... [the output was truncated (use n=-1 to disable)]
[ CPUByteType{325862} ]

We’ve made it so autocomplete works properly, such which you could interactively discover the accessible
operators utilizing jit_ops$ and urgent to set off RStudio’s autocomplete.

Other small enhancements

This launch additionally provides many small enhancements that make torch extra intuitive:

• You can now specify the tensor dtype utilizing a string, eg: torch_randn(3, dtype = "float64"). (Previously you needed to specify the dtype utilizing a torch operate, akin to torch_float64()).

  torch_randn(3, dtype = "float64")

  torch_tensor
  -1.0919
   1.3140
   1.3559
  [ CPUDoubleType{3} ]

• You can now use with_device() and local_device() to briefly modify the gadget
  on which tensors are created. Before, you had to make use of gadget in every tensor
  creation operate name. This permits for initializing a module on a selected gadget:

  with_device(gadget="mps", {
    linear <- nn_linear(10, 1)
  })
  linear$weight$gadget

  torch_device(sort='mps', index=0)

• It’s now potential to briefly modify the torch seed, which makes creating
  reproducible packages simpler.

  with_torch_manual_seed(seed = 1, {
    torch_randn(1)
  })

  torch_tensor
   0.6614
  [ CPUFloatType{1} ]

Thank you to all contributors to the torch ecosystem. This work wouldn’t be potential with out
all of the useful points opened, PRs you created, and your laborious work.

If you’re new to torch and need to be taught extra, we extremely suggest the lately introduced ebook ‘Deep Learning and Scientific Computing with R torch’.

If you need to begin contributing to torch, be happy to succeed in out on GitHub and see our contributing information.

The full changelog for this launch will be discovered right here.

Photo by Ian Schneider on Unsplash

Reuse

Text and figures are licensed below Creative Commons Attribution CC BY 4.0. The figures which have been reused from different sources do not fall below this license and will be acknowledged by a be aware of their caption: “Figure from …”.

Citation

For attribution, please cite this work as

Falbel (2023, June 7). Posit AI Blog: torch 0.11.0. Retrieved from https://blogs.rstudio.com/tensorflow/posts/2023-06-07-torch-0-11/

BibTeX quotation

@misc{torch-0-11-0,
  creator = {Falbel, Daniel},
  title = {Posit AI Blog: torch 0.11.0},
  url = {https://blogs.rstudio.com/tensorflow/posts/2023-06-07-torch-0-11/},
  yr = {2023}
}