Emacs, Perl, and Jupyter

Run Perl incrementally from an Emacs Org file using Devel::IPerl and emacs-jupyter, following the steps below.

Emacs is a very popular editor with a long history, going back to 1976. It is an extensible, customizable, self-documenting real-time display editor, It has tools for compiling, running and debuging programs, is a file manager, a news and email reader, etc. It is written in emacs-lisp and includes a lisp interpreter, so it is easily extended. Jupyter is a notebook system that allows editing notes intermixed with program fragments written in one of several several languages, running them and examining the results, exporting them in several formats, and all from a web interface. Perl is a very expressive, highly capable, feature-rich computer language, appropriate for both rapid prototyping and large scale development projects. Org mode is an Emacs mode for keeping and organizing notes, authoring documents, computational notebooks, literate programming, maintaining to-do lists, planning projects, and more, in a fast and effective plain text system. It allows intermixing text with computer codes in many languages, and allows running the codes and displaying their results, including graphics and tables. Org files may be exported to a large number of formats, such as publication ready latex and pdf documents, markdown, html, plain text, etc. It may seem similiar to Jupyter, but it is more general purpose than development of code, it runs within a very mature and powerful editor, and it allows mixing codes of different languages in the same document. In particular, loading the language package ob-perl allows Org files to contain and run Perl codes. Nevertheless, Perl code sections have to be self-contained in order to be run, or program fragments that may be joined into a full program by tangling or by using the literate programing tool noweb before they are run. They may not be run and tested incrementally, as other languages can. I had been thinking of modifying ob-perl to allow running Perl interactive sessions from an Org file. Nevertheless, I found that Zakariyya Mughal has developed a package Devel::IPerl with a Perl language kernel and an interface to Jupyter, allowing interactive Perl sessions to be run within Jupyter. I also found an emacs package jupyter that allows running jupyter sessions from Org code blocks. With these tools I can now run Perl program fragments interactively from within an Org session through a Jupyter session.

The steps to accomplish this were:

  1. Install Jupyter from the package manager. My system is Debian so I used apt:

    sudo apt install jupyter
    
  2. Install Devel::IPerl from CPAN. I manage my Perl installation through perlbrew and cpanm:

    cpanm Devel::IPerl
    
  3. Install ob-async from the emacs package manager, running package-list-packages, looking for ob-async, marking it for installation and executing the marked actions, using the Package menu.
  4. Similarly, install jupyter from the emacs package manager.
  5. Add some lines to the .emacs initialization file:

    ;; load ob-async to execute code asynchronously
    (require 'ob-async)
    ; jupyter has its own async
    (setq ob-async-no-async-languages-alist '("jupyter-python" "jupyter-perl"))
    
  6. I believe those lines should go before loading the babel languages.

    (org-babel-do-load-languages
     'org-babel-load-languages
     '(
       ...  ; All your previous previous values
       (jupyter . t) ; add jupyter
       ))
    
  7. Restart Emacs (or run the elisp codes above from a running Emacs).
  8. Then from an org file I can do things like the following:

    #+begin_src jupyter-perl :session ip :exports both :results output
    use v5.36;
    my $x=1;
    say $x;
    #+end_src
    

    This is rendered as follows when exported:

    use v5.36;
    my $x=1;
    say $x;
    

    I can run the code with the key sequence C-cC-c with point (the cursor) anyplace within the code block. The result is

    1
    

    Now I can add a second block and run it:

    $x*=2;  # modify a variable declared in another block
    say $x;
    

    Results:

    2
    

    Notice that when I ran the second code block, the value of the lexical variable $x was already known. I can modify it again:

    $x*=3;  # modify again a variable declared in another block
    say $x;
    

    Results:

    6
    

    I can add to the #+begin_src lines of the three blocks above a header argument such as :tangle program.pl, so when I run tangle C-c C-v t the three are combined into a single monolithic program and saved in a file program.pl. Furthermore, if I add the header :shebang #!/usr/bin/env perl the file will have #!/usr/bin/env perl as its first line and it will have the executable bit turned on. Then I can run the program with code as:

    #+begin_src bash :results output :exports both :cache yes
    ./program.pl
    #+end_src
    

    rendered as

    ./program.pl
    

    Results:

    1
    2
    6
    

    I can add :async yes to the header arguments in order to run the code block in a non-blocking mode and continue editing without waiting for the result, as in

    #+begin_src jupyter-perl :session ip :exports both :results output :async yes
    say expensive_calculation();
    sub expensive_calculation(){
        sleep(10); # fake the effort for 10s
        return 42; # what else?
    }
    #+end_src
    

    rendered as:

    say expensive_calculation();
     sub expensive_calculation(){
         sleep(10); # fake the effort
         return 42; # what else?
     }
    

    The result is

    5f03a87f-62c0-49da-8a7b-6204caa490b9
    

    for ten seconds, while I may continue editing, and then it becomes the actual result

    42
    

    I may run graphical programs as:

    use PDL;
    use PDL::Graphics::Gnuplot;
    my $t=zeroes(100)->xlinvals(0,24);
    gplot(with=>"lines", $t, $t->sin, with=>"lines", $t, $t->cos);
    

    From the gnuplot window I can save the plot, or I can set the gnuplot terminal to some file, say filename.png, and insert the graphics into the Org file with the code

    [[./filename.png]]
    

    Results: img

I hope I didn’t forget any steps and that it work for others.

Thanks to Zakariyya Mughal, author of Devel::IPerl, for his advice and his work.

Written on August 8, 2023