Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Add reactor-cpp as a submodule and reorganize the C++ build process #971

Merged
merged 31 commits into from
Feb 28, 2022
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
1dbb230
add reactor-cpp as a submodule
cmnrd Feb 15, 2022
46d6431
simplify copyFileFromClassPath
cmnrd Feb 16, 2022
61e1bcb
copy reactor-cpp to the src-gen directory
cmnrd Feb 16, 2022
632f16f
generate a root cmake script and a specialized one
cmnrd Feb 16, 2022
755caf1
pass various parameters on to reactor-cpp
cmnrd Feb 16, 2022
fb18037
update compilation command
cmnrd Feb 16, 2022
c0b62bd
support multiple versions of reactor-cpp side by side
cmnrd Feb 17, 2022
4554be6
Avoid unnecessary file overwrites in copyDirectoryFromClassPath
cmnrd Feb 17, 2022
b6c1d51
support the runtime-version property again
cmnrd Feb 17, 2022
696a74a
support external-runtime-path
cmnrd Feb 17, 2022
7552a71
only build and install the currently active target
cmnrd Feb 17, 2022
4e06bb4
don't overwrite generated files if they are unchanged
cmnrd Feb 17, 2022
69cd7de
also add a CMakeLists.txt to any subdirectories in src-gen
cmnrd Feb 18, 2022
7fe1404
update reactor-cpp version in test script
cmnrd Feb 18, 2022
cf2bbd2
fix benchmark CI workflow
cmnrd Feb 18, 2022
ae0e480
use the updated CI workflow
cmnrd Feb 18, 2022
d0463f2
only generate and build reactor-cpp if it is needed
cmnrd Feb 18, 2022
eab0050
add messages for debugging macOS build errors
cmnrd Feb 18, 2022
304a96a
runtime version txt file is not needed anymore
cmnrd Feb 18, 2022
7117319
fix C++ compilation on macOS
cmnrd Feb 18, 2022
0340b6c
C++ LSP tests pass locally.
petervdonovan Feb 20, 2022
9fd4c91
C++: Remove special case that is now unnecessary.
petervdonovan Feb 20, 2022
3b73c50
Apply "skipIfUnchanged" optimization to target language validator.
petervdonovan Feb 21, 2022
64b6e0a
correctly copy directories from a bundle (i.e. from within epoch)
cmnrd Feb 23, 2022
644ed18
cleanup: avoid code duplication, change name for copying input streams
cmnrd Feb 25, 2022
c09c044
make sure streams are always closed
cmnrd Feb 25, 2022
1de38fb
cleanup
cmnrd Feb 25, 2022
0d322d4
Apply suggestions from code review
lhstrh Feb 25, 2022
2a4ed86
Addressed comment from review
lhstrh Feb 25, 2022
0493e45
change default lib name to reactor-cpp-default
cmnrd Feb 28, 2022
124e2c2
Update ci worklfow to checkout correct reactor-cpp
cmnrd Feb 28, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/scripts/test-lfc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ bin/lfc --output-path . test/C/src/Minimal.lf
# --runtime-version <arg> Specify the version of the runtime
# library used for compiling LF
# programs.
bin/lfc --runtime-version 26e6e641916924eae2e83bbf40cbc9b933414310 test/Cpp/src/Minimal.lf
bin/lfc --runtime-version ca216ccc3da5ecff0e8013f75e275d7acac099de test/Cpp/src/Minimal.lf

# -t,--threads Specify the default number of threads.
bin/lfc -t 2 test/C/src/Minimal.lf
Expand Down
11 changes: 7 additions & 4 deletions .github/workflows/benchmark-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,17 @@ jobs:
run: |
python3 benchmark/runner/run_benchmark.py -m test_mode=True iterations=1 benchmark="glob(*)" target=lf-c,lf-c-unthreaded
if: ${{ inputs.target == 'C' }}
- name: Setup C++ build environment
- name: Compile reactor-cpp once and reuse for all benchmarks
run: |
./bin/lfc test/Cpp/src/Minimal.lf
echo "LD_LIBRARY_PATH=$GITHUB_WORKSPACE/test/Cpp/lib" >> $GITHUB_ENV
mkdir -p reactor-cpp/build
cd reactor-cpp/build
cmake -DCMAKE_INSTALL_PREFIX=../install ../../org.lflang/src/lib/cpp/reactor-cpp
make install
echo "LD_LIBRARY_PATH=$GITHUB_WORKSPACE/reactor-cpp/install/lib" >> $GITHUB_ENV
if: ${{ inputs.target == 'Cpp' }}
- name: Test C++ benchmarks
run: |
python3 benchmark/runner/run_benchmark.py -m test_mode=True iterations=1 benchmark="glob(*)" target=lf-cpp iterations=1 target.params.extra_args="[--external-runtime-path, ${GITHUB_WORKSPACE}/test/Cpp]"
python3 benchmark/runner/run_benchmark.py -m test_mode=True iterations=1 benchmark="glob(*)" target=lf-cpp iterations=1 target.params.extra_args="[--external-runtime-path, ${GITHUB_WORKSPACE}/reactor-cpp/install]"
if: ${{ inputs.target == 'Cpp' }}
- name: Setup Rust
uses: ATiltedTree/setup-rust@v1
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:

# Run the C++ benchmark tests.
cpp-benchmark-tests:
uses: lf-lang/lingua-franca/.github/workflows/benchmark-tests.yml@master
uses: lf-lang/lingua-franca/.github/workflows/benchmark-tests.yml@cpp-submodule
with:
target: 'Cpp'

Expand Down
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,6 @@
[submodule "org.lflang/src/lib/py/reactor-c-py"]
path = org.lflang/src/lib/py/reactor-c-py
url = https://github.com/lf-lang/reactor-c-py.git
[submodule "org.lflang/src/lib/cpp/reactor-cpp"]
path = org.lflang/src/lib/cpp/reactor-cpp
url = https://github.com/lf-lang/reactor-cpp
1 change: 1 addition & 0 deletions org.lflang/src/lib/cpp/reactor-cpp
Submodule reactor-cpp added at 604c97
155 changes: 139 additions & 16 deletions org.lflang/src/org/lflang/FileConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,20 @@
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.net.JarURLConnection;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.Arrays;
import java.util.Comparator;
import java.util.Enumeration;
import java.util.LinkedList;
import java.util.List;
import java.util.function.Consumer;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.stream.Collectors;
import java.util.stream.Stream;

Expand Down Expand Up @@ -422,13 +428,72 @@ public static void copyDirectory(Path src, Path dest) throws IOException {
/**
* Copy a given file from 'source' to 'destination'.
*
* This also creates new directories for any directories on the destination
* path that do not yet exist.
*
* @param source The source file path.
* @param destination The destination file path.
* @param skipIfUnchanged If true, don't overwrite the destination file if its content would not be changed
* @throws IOException if copy fails.
*/
public static void copyFile(Path source, Path destination) throws IOException {
public static void copyFile(Path source, Path destination, boolean skipIfUnchanged) throws IOException {
Files.createDirectories(destination);
cmnrd marked this conversation as resolved.
Show resolved Hide resolved
if(skipIfUnchanged && Files.isRegularFile(destination)) {
if (Arrays.equals(Files.readAllBytes(source), Files.readAllBytes(destination))) {
return;
}
}
Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING);
}

/**
* Copy a given input stream to a destination file.
*
* This also creates new directories for any directories on the destination
* path that do not yet exist.
*
* @param source The source input stream.
* @param destination The destination file path.
* @param skipIfUnchanged If true, don't overwrite the destination file if its content would not be changed
* @throws IOException if copy fails.
*/
public static void copyFile(InputStream source, Path destination, boolean skipIfUnchanged) throws IOException {
Files.createDirectories(destination.getParent());
if(skipIfUnchanged && Files.isRegularFile(destination)) {
if (Arrays.equals(source.readAllBytes(), Files.readAllBytes(destination))) {
return;
}
}
Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING);
}

/**
* Copy a given file from 'source' to 'destination'.
*
* This also creates new directories for any directories on the destination
* path that do not yet exist.
*
* @param source The source file path.
* @param destination The destination file path.
* @throws IOException if copy fails.
*/
public static void copyFile(Path source, Path destination) throws IOException {
copyFile(source, destination, false);
}

/**
* Copy a given input stream to a destination file.
*
* This also creates new directories for any directories on the destination
* path that do not yet exist.
*
* @param source The source file path.
* @param destination The destination file path.
* @throws IOException if copy fails.
*/
public static void copyFile(InputStream source, Path destination) throws IOException {
copyFile(source, destination, false);
}

/**
* Lookup a file in the classpath and copy its contents to a destination path
Expand All @@ -439,32 +504,90 @@ public static void copyFile(Path source, Path destination) throws IOException {
*
* @param source The source file as a path relative to the classpath.
* @param destination The file system path that the source file is copied to.
* @param skipIfUnchanged If true, don't overwrite the destination file if its content would not be changed
* @throws IOException If the given source cannot be copied.
*/
public void copyFileFromClassPath(String source, Path destination) throws IOException {
public void copyFileFromClassPath(final String source, final Path destination, final boolean skipIfUnchanged) throws IOException {
InputStream sourceStream = this.getClass().getResourceAsStream(source);

// Copy the file.
try (sourceStream) {
cmnrd marked this conversation as resolved.
Show resolved Hide resolved
if (sourceStream == null) {
throw new IOException(
"A required target resource could not be found: " + source + "\n" +
"Perhaps a git submodule is missing or not up to date.\n" +
"See https://github.com/icyphy/lingua-franca/wiki/downloading-and-building#clone-the-lingua-franca-repository.\n"
+
"Also try to refresh and clean the project explorer if working from eclipse.");
}
if (sourceStream == null) {
throw new IOException(
"A required target resource could not be found: " + source + "\n" +
"Perhaps a git submodule is missing or not up to date.\n" +
lhstrh marked this conversation as resolved.
Show resolved Hide resolved
"See https://github.com/icyphy/lingua-franca/wiki/downloading-and-building#clone-the-lingua-franca-repository.\n"
lhstrh marked this conversation as resolved.
Show resolved Hide resolved
+
"Also try to refresh and clean the project explorer if working from eclipse.");
lhstrh marked this conversation as resolved.
Show resolved Hide resolved
} else {
// Make sure the directory exists
//noinspection ResultOfMethodCallIgnored
destination.toFile().getParentFile().mkdirs();
cmnrd marked this conversation as resolved.
Show resolved Hide resolved
copyFile(sourceStream, destination, skipIfUnchanged);
}
}

Files.copy(sourceStream, destination, StandardCopyOption.REPLACE_EXISTING);
} catch (IOException ex) {
/**
* Lookup a file in the classpath and copy its contents to a destination path
* in the filesystem.
*
* This also creates new directories for any directories on the destination
* path that do not yet exist.
*
* @param source The source file as a path relative to the classpath.
* @param destination The file system path that the source file is copied to.
* @throws IOException If the given source cannot be copied.
*/
public void copyFileFromClassPath(final String source, final Path destination) throws IOException {
copyFileFromClassPath(source, destination, false);
}

/**
* Lookup a directory in the classpath and copy its contents to a destination path
* in the filesystem.
*
* This also creates new directories for any directories on the destination
* path that do not yet exist.
*
* @param source The source directory as a path relative to the classpath.
* @param destination The file system path that the source directory is copied to.
* @param skipIfUnchanged If true, don't overwrite the file if its content would not be changed
* @throws IOException If the given source cannot be copied.
*/
public void copyDirectoryFromClassPath(final String source, final Path destination, final boolean skipIfUnchanged) throws IOException {
cmnrd marked this conversation as resolved.
Show resolved Hide resolved
final URL resource = getClass().getResource(source);
if (resource == null) {
throw new IOException(
"A required target resource could not be copied: " + source + "\n" +
"A required target resource could not be found: " + source + "\n" +
"Perhaps a git submodule is missing or not up to date.\n" +
"See https://github.com/icyphy/lingua-franca/wiki/downloading-and-building#clone-the-lingua-franca-repository.",
ex);
"See https://github.com/icyphy/lingua-franca/wiki/downloading-and-building#clone-the-lingua-franca-repository.\n"
lhstrh marked this conversation as resolved.
Show resolved Hide resolved
+
"Also try to refresh and clean the project explorer if working from eclipse.");
}

// create a connection to the jar
final JarURLConnection connection = (JarURLConnection) resource.openConnection();
final JarFile jar = connection.getJarFile();
final String connectionEntryName = connection.getEntryName();

// Iterate all entries in the jar file.
for (Enumeration<JarEntry> e = jar.entries(); e.hasMoreElements();) {
final JarEntry entry = e.nextElement();
final String entryName = entry.getName();

// Extract files only if they match the given source path.
if (entryName.startsWith(connectionEntryName)) {
String filename = entry.getName().substring(connectionEntryName.length() + 1);
Path currentFile = destination.resolve(filename);

if (entry.isDirectory()) {
//noinspection ResultOfMethodCallIgnored
currentFile.toFile().mkdirs();
} else {
InputStream is = jar.getInputStream(entry);
copyFile(is, currentFile, skipIfUnchanged);
is.close();
}
}
}
}

Expand Down
23 changes: 20 additions & 3 deletions org.lflang/src/org/lflang/generator/JavaGeneratorUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
Expand Down Expand Up @@ -312,19 +313,35 @@ public static LFResource getLFResource(
* Write text to a file.
* @param text The text to be written.
* @param path The file to write the code to.
* @param skipIfUnchanged If true, don't overwrite the destination file if its content would not be changed
*/
public static void writeToFile(String text, Path path) throws IOException {
path.getParent().toFile().mkdirs();
public static void writeToFile(String text, Path path, boolean skipIfUnchanged) throws IOException {
Files.createDirectories(path.getParent());
final byte[] bytes = text.getBytes();
if (skipIfUnchanged && Files.isRegularFile(path)) {
if (Arrays.equals(bytes, Files.readAllBytes(path))) {
return;
}
}
Files.write(path, text.getBytes());
}

/**
* Write text to a file.
* @param text The text to be written.
* @param path The file to write the code to.
*/
public static void writeToFile(String text, Path path) throws IOException {
writeToFile(text, path, false);
}

/**
* Write text to a file.
* @param text The text to be written.
* @param path The file to write the code to.
*/
public static void writeToFile(CharSequence text, Path path) throws IOException {
writeToFile(text.toString(), path);
writeToFile(text.toString(), path, false);
}

/**
Expand Down
Loading