J/Link Tutorial: Reading a URL

download notebookDownload this example as a Mathematica notebook.

An obvious use of Java is reading files off the internet. These files can be Mathematica packages (i.e., .m files) to be read into the kernel or notebooks (.nb files) to be opened in the front end. As a common foundation for these two operations, first a function must be created that takes an arbitrary file specified by a URL and copies it to a local temporary file. Java has a class called java.net.URL that represents a URL and allows data to be downloaded. In particular, there is an openStream method that returns a standard InputStream, which can be easily read from. Later, embellishments can be added that cause the file to be read into the kernel or opened as a notebook in the front end.

Starting Out

The following pseudocode was written to demonstrate what will be needed in the code.

GetURL[url_String] :=
    open a temporary local file
    create a new Java URL object
    create a Java input stream from the URL object
    read from the URL, writing the data to the local file
    close the stream
    close the local file
    release the Java objects created
    return the local filename

Here is the fundamental code that J/Link needs in order to read files from the internet.


There is one interesting wrinkle in the code. You need to use the read method of the InputStream you get back from openStream. The read method is declared as public int read(byte[] b). The read method does not return the array of data; it takes the array as an argument and fills it. A Java method that takes an array of bytes would typically be called from Mathematica with a list of integers. If you create a list of 5000 integers, 0s for example, and send that as the argument to read, what would happen? An array of this data would be created in Java and passed to read. The read method would then fill the array with data, overwriting the 0s, and return the number of bytes read. This method works fine except for one problem: you cannot get the data back into Mathematica! You have no way of retrieving the array that was created from the list of numbers you sent.

The solution to this problem is to create an array of bytes on the Java side with JavaNew. Then you have a reference to the array that will be filled by read. You can pass this reference (buf in the code above) to the read method instead of a Mathematica list of numbers. Recall that all Java methods that take arrays can be called from Mathematica with either a Mathematica list or a reference to an array on the Java side. The same is true for strings.

To create a Java array object, you need to use JavaNew with the class name as the first argument and the length as the second argument. The actual Java class name for an array of bytes is [B, and J/Link makes no attempt to hide these cryptic array class names behind a prettier interface. To create a multidimensional array, pass a list of integers as the second argument. For example, JavaNew["[[I", {3, 4}] creates the array int[3][4].

When the array has been filled by read, calling Val forces the array reference buf to be converted into its Mathematica value representation, which is a list of integers. These numbers are then converted to a string via FromCharacterCode and written to the file. Section 1.2.4 of the J/Link User Manual discusses Val and other issues concerning how J/Link handles "by value" and "by reference" representations of Java objects.

Now that you have the GetURL function, you can easily write functions that read files directly off the internet into the kernel or front end. In the following example Get is overloaded to treat strings that begin with "http://" specially:


To open notebooks off the internet, you can create the function NotebookOpenURL. It would be nice to add a definition for the built-in function NotebookOpen that works for URLs, as was done above with Get. However, it takes a bit of a hack to get the definition used ahead of the standard definitions, so a more straightforward method is used here.


Now try it. A notebook window will open.


Final Code

The following example combines all the above elements for J/Link to read files from the internet.