New Julia Project

Author
Affiliation

Callum Arnold-Leps

Pennsylvania State University

Published

May 26, 2023

First Time Julia Setup

Note

You only have to follow these steps the first time after you download a new version of Julia and set it as your working version.

Now that we have a better understanding of both Julia and the VSCode user interface, let’s create a new Julia project.

First, open a fresh VSCode window. If you already happen to be in a workspace (check in the “Explorer” panel), you can click on the “File” menu in the top left of the VSCode window, and then click “New Window”. Now, open the Command Palette (keyboard shortcut Cmd + Shift + P on Mac or Ctrl + Shift + P on Windows or Linux) and type “Julia: Start REPL”. Doing this should open up the integrated terminal with the Julia REPL running. Type pwd() in the REPL and hit enter. It should return your home directory (i.e. /Users/<your-username> on Mac or C:\Users\<your-username> on Windows). From here we want to add a package called DrWatson to our Julia installation. To do this, type ] in the REPL and hit enter. This enters the Pkg REPL, which is a special REPL for managing Julia packages. Here you should see something like:

(@v1.10) pkg>

Next, type add DrWatson and hit enter. This will add the DrWatson package to your Julia installation, and will allow us to easily create new Julia projects in a consistent way that works well for epidemiology projects (among others).

Note

If you’re still wondering why we’re bothering with the DrWatson package, you can read more about it below.

Creating a Folder for Your Projects

The first thing we need to do is create a folder that will house all our individual project folders. To try and make your life easy, I would recommend you create this folder in your home directory, and call it Repos/ (or something similar). You can do this with Finder on Mac or File Explorer on Windows.

Creating a Julia Project

Now let’s create our first Julia project folder. To do this, go back to the Julia Pkg REPL and hit backspace. This will return you to the normal Julia REPL. Now we need to navigate to the folder you just created. To do this, type cd("path/to/your/project/folder") and hit enter.

From here, type using DrWatson and hit enter. This will load the DrWatson package into your Julia session so you can use its functions.

To create the project folder, type initialize_project("MyFirstProject") and hit enter. This will create a new project folder MyFirstProject/ in the folder you navigated to earlier e.g., /Users/username/Documents/Repos/MyFirstProject/.

Now you have a project folder, let’s open it in VSCode. In VSCode, click on the “File” menu in the top left of the VSCode window, and then click “Open Folder…”. From here, navigate to the project folder you just created and click “Open”. You VSCode window should refresh and you should now see the name of your project folder in the top center of the VSCode window (as well as in the top left of the “Explorer” panel).

Now, if you try to open the Pkg REPL again, you will see something different. Instead it’ll say:

(MyFirstProject) pkg>

You’re ready to start adding packages to your project! I would recommend starting with following packages:

  • Revise
  • DrWatson
  • DataFrames
  • DataFramesMeta
  • CSV
  • IJulia
  • JuliaFormatter
  • CairoMakie

As before, to add a package, type add <package-name> and hit enter (you can add multiple packages at once by separating them with spaces e.g. add <package-1> <package-2>).

Project Files

When you create a new Julia project, it creates two files in the project folder:

  • Project.toml: this is a TOML file that contains the names and versions of all the packages you have added to your project
  • Manifest.toml: this is a TOML file that contains the names and versions of all the packages you have added to your project, as well as the names and versions of all the packages that those packages depend on. It is automatically generated by Julia, and you should not edit it manually.

Edit the Project.toml file to add your name in the authors field at the top of the file (you will need to create this), so it looks something like this:

name = "MyFirstProject"
authors = ["My Name"]

[deps]
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
DataFramesMeta = "1313f7d8-7da2-5740-9ea0-a2ca25f37964"
DrWatson = "634d3b9d-ee7a-5ddf-bec9-22491ea816e1"
IJulia = "7073ff75-c697-5162-941a-fcdaad2a7d2a"
JuliaFormatter = "98e50ef6-434e-11e9-1051-2b60c6c9e899"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"

Creating & Running a Julia Script

To create a Julia script in your project, click on the “File” menu in the top left of the VSCode window, and then click “New File”. Alternatively, you could use the keyboard shortcut Cmd + N on Mac or Ctrl + N on Windows or Linux. This will open a new file in the VSCode editor, which you can then save. Given we are using DrWatson, it is also good practice to save it in the scripts/ folder. You can save it with any file name you like, but it is good practice to save it with the .jl extension e.g., scripts/myscript.jl.

At this point, you can start writing your Julia code. Just make sure that the first two lines of the script are:

using DrWatson
quickactivate("MyFirstProject")

Miscellaneous

DrWatson Project Structure

It may not be immediately obvious why we need to use DrWatson to create our Julia projects. In fact, we don’t need to use DrWatson to create our Julia projects, but it does make our lives a lot easier as it provides a consistent project structure that is easy to navigate.

Firstly, it creates the following folders and files in your project folder (copied from the DrWatson documentation):

│projectdir          <- Project's main folder. It is initialized as a Git
│                       repository with a reasonable .gitignore file.
│
├── _research        <- WIP scripts, code, notes, comments,
│   |                   to-dos and anything in an alpha state.
│   └── tmp          <- Temporary data folder.
│
├── data             <- **Immutable and add-only!**
│
├── plots            <- Self-explanatory.
├── notebooks        <- Jupyter, Weave or any other mixed media notebooks.
│
├── papers           <- Scientific papers resulting from the project.
│
├── scripts          <- Various scripts, e.g. simulations, plotting, analysis,
│   │                   The scripts use the `src` folder for their base code.
│   └── intro.jl     <- Simple file that uses DrWatson and uses its greeting.
│
├── src              <- Source code for use in this project. Contains functions,
│                       structures and modules that are used throughout
│                       the project and in multiple scripts.
│
├── README.md        <- Optional top-level README for anyone using this project.
├── .gitignore       <- by default ignores _research, data, plots, videos,
│                       notebooks and latex-compilation related files.
│
├── Manifest.toml    <- Contains full list of exact package versions used currently.
└── Project.toml     <- Main project file, allows activation and installation.
                        Includes DrWatson by default.

In addition to the {DrWatson} produced directories, I like to create an out/ directory where all analysis outputs are created, e.g., the sims shown in the structure above, rather than saving them to the data/ directory, as {DrWatson} suggests. The reason for doing this is that it provides a very clear separation of input and output files, where inputs should never be modified, and outputs are files that should always be able to deleted and recreated using the code in the scripts. As a result, the final structure of the project looks like this:

│projectdir          <- Project's main folder. It is initialized as a Git
│                       repository with a reasonable .gitignore file.
│
├── _research        <- WIP scripts, code, notes, comments,
│   |                   to-dos and anything in an alpha state.
│   └── tmp          <- Temporary data folder.
│
├── data             <- **Immutable and add-only!**
│   └── exp_raw      <- Raw experimental data.
│
├── out              <- **Immutable and add-only!**
│   ├── exp_pro      <- Data from processing experiments.
│   └── sims         <- Data resulting directly from simulations.
│
├── plots            <- Self-explanatory.
├── notebooks        <- Jupyter, Weave or any other mixed media notebooks.
│
├── papers           <- Scientific papers resulting from the project.
│
├── scripts          <- Various scripts, e.g. simulations, plotting, analysis,
│   │                   The scripts use the `src` folder for their base code.
│   └── intro.jl     <- Simple file that uses DrWatson and uses its greeting.
│
├── src              <- Source code for use in this project. Contains functions,
│                       structures and modules that are used throughout
│                       the project and in multiple scripts.
│
├── README.md        <- Optional top-level README for anyone using this project.
├── .gitignore       <- by default ignores _research, data, plots, videos,
│                       notebooks and latex-compilation related files.
│
├── Manifest.toml    <- Contains full list of exact package versions used currently.
└── Project.toml     <- Main project file, allows activation and installation.
                        Includes DrWatson by default.

Above providing us with a useful project structure, DrWatson also provides us with a number of very helpful functions.

For one, it provides us with the quickactivate() function that allows us to activate our project and environment when we’re working in a script, so we can easily import all the packages we need, regardless of where the script is saved. This means that we don’t need to worry about activating our project in the REPL before we start working in a script. This may not sound meaningful, but it’s very easy to forget to activate your project in the REPL before you start working in a script, and then you end up with a bunch of errors because you haven’t actually imported the packages you need. To use this, make sure your first two lines of your script are always:

using DrWatson
quickactivate("MyFirstProject")

And secondly, it provides us with convenient directory paths to the folders in our project. For example, if we want to access the data folder in our project, we can use the datadir() function, which will return the path to the data folder. Similarly, if we wanted to save a plot to the plots folder, we could use save(plotsdir("plot_object.png"), plot_object) function to save the plot in plot_object to the plots folder.

More details about the DrWatson project structure and associated functions can be found here.

Naming Files

When naming files and folders, it is important to be consistent and descriptive. Ideally the name will contain no spaces, and you can use dashes to separate words. See this presentation by Jenny Bryan for more information on naming files and folders, but in short:

  • KISS (Keep It Simple Stupid): use simple and consistent file names
    • It needs to be machine readable
    • It needs to be human readable
    • It needs to order well in a directory
    • No special characters and no spaces!
  • Use YYYY-MM-DD date format
  • Use - to delimit words and _ to delimit sections
    • i.e. 2019-01-19_my-data.csv
  • Left-pad numbers
    • i.e. 01_my-data.csv vs 1_my-data.csv
    • If you don’t, file orders get messed up when you get to double-digits