You can program your data processing tasks using the
ScalaTask
. For instance the following
workflow sums all the elements of an array using a
ScalaTask
and displays the results. To get more
details on the hook part you can check the doc on
hooks.
val array = Val[Array[Double]]
val result = Val[Double]
val sum = ScalaTask("val result = array.sum") set (
inputs += array,
outputs += result,
array := Array(8.0, 9.0, 10.0)
)
(sum hook ToStringHook())
Multiple
ScalaTasks can be joint to compose a workflow. Lets imagine that you want to
perform an expensive computation on every element of an array. For the sake of simplicity, the "expensive computation"
is here a multiplication by 2.
val element = Val[Double]
val multiplied = Val[Double]
val result = Val[Double]
val expensive = ScalaTask("val multiplied = element * 2") set (
inputs += element,
outputs += multiplied
)
val exploration = ExplorationTask(element in List(8.0, 9.0, 10.0))
val sum = ScalaTask("val result = multiplied.sum") set (
inputs += multiplied.toArray,
outputs += result
)
exploration -< expensive >- (sum hook ToStringHook())
The execution of this workflow can be distributed using OpenMOLE's environments. Check the page dedicated to
environments to learn more on this process.
OpenMOLE makes it simple to include your own Java code in a workflow. A Java program can be encapsulated in a task of
a workflow. The task should be a ScalaTask and the Java program should be packaged as a JAR archive passed to the
ScalaTask through the
libraries
parameter.
Let us consider the simple code
Hello.java in a directory named
hello (to respect Java's package structure):
package hello;
public class Hello {
public static void run(int arg) {
System.out.println("Hello from Java! " + arg);
}
}
We compile the code and generate the JAR file as follows:
mkdir hello
mv Hello.java hello
cd hello
javac Hello.java
cd ..
jar cvf Hello.jar hello
You should upload the Hello.jar file in a folder in OpenMOLE. Then you can call the Java program from a ScalaTask with the following OpenMOLE code:
val proto1 = Val[Int]
val explo = ExplorationTask(proto1 in (0 until 10))
//Defines the task to perform the hello function
val javaTask = ScalaTask("hello.Hello.run(proto1)") set (
libraries += workDirectory / "Hello.jar",
inputs += proto1
)
explo -< javaTask
The output should look like that (the order in which the lines are printed might be different in your case):
Hello from Java! 0
Hello from Java! 1
Hello from Java! 2
Hello from Java! 3
Hello from Java! 4
Hello from Java! 5
Hello from Java! 6
Hello from Java! 7
Hello from Java! 8
Hello from Java! 9
Hello from Java! 10
In the general case a task is used to compute some output values depending on some input values. To illustrate
that, let's consider another Java code:
package hello;
public class Hello {
public static double[] run(double arg1, double arg2) {
return double[]{arg1 * 10, arg2 * 10};
}
}
Once you have packaged this code in the same way as before, it can be explored in OpenMOLE:
val arg1 = Val[Double]
val arg2 = Val[Double]
val out1 = Val[Double]
val out2 = Val[Double]
val explo = ExplorationTask(
(arg1 in (0.0 to 10.0 by 1.0)) x
(arg2 in (0.0 until 10.0 by 1.0))
)
val javaTask = ScalaTask("hello.Array(out1, out2) = Hello.run(arg1, arg2)") set (
libraries += workDirectory / "Hello.jar",
inputs += (arg1, arg2),
outputs += (arg1, arg2, out1, out2)
)
// save the result in a CSV file
val csvHook = AppendToCSVFileHook(workDirectory / "result.csv")
explo -< (javaTask hook csvHook)
You can compile it and package as "Hello.jar" in the same manner as the previous example. This workflow explores the 2 arguments of the
hello function and saves the results in a CSV file.
Let's consider another "hello world" code Hello.java. This program reads the content of a file and writes it to
another file.
package hello;
import java.io.*;
public class Hello {
public static void run(int arg, File input, File output) throws IOException {
//Read the input file
String content = readFileAsString(input);
PrintStream myStream = new PrintStream(new FileOutputStream(output));
try {
myStream.println(content + " " + arg);
} finally {
myStream.close();
}
}
private static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
BufferedInputStream f = null;
try {
f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
} finally {
if (f != null) try { f.close(); } catch (IOException ignored) { }
}
return new String(buffer);
}
}
You can compile it and package as "Hello.jar" in the same manner as the previous example. This Java program can be explored with OpenMOLE using the following script:
val proto1 = Val[Int]
val inputFile = Val[File]
val outputFile = Val[File]
val explo = ExplorationTask(proto1 in (0 to 10))
//Defines the scala task as a launcher of the hello executable
val javaTask =
ScalaTask("val outputFile = newFile(); hello.Hello.run(proto1, inputFile, outputFile)") set (
libraries += workDirectory / "Hello.jar",
inputs += (proto1, inputFile),
outputs += (proto1, outputFile),
inputFile := workDirectory / "input.txt"
)
//Save the output file locally
val copyHook =
CopyFileHook(
outputFile,
workDirectory / "out-${proto1}.txt"
)
explo -< (javaTask hook copyHook)
For this example to work you should create a file named "input.txt" in the work directory of your project.
This tutorial works for simple Java programs. For more ambitious developments, you should consider embedding
your code in an
OpenMOLE plugin.