Installing on OS X - linking issue

[gdtroszak]

I've managed to install the shadow compiler on OS X using the Linux instructions. However, when I try to compile the "Hello, World!" tutorial shadowc HelloWorld.shadow I get the following error;

FILE DEPENDENCY ERROR: Cannot run program "llvm-link": error=2, No such file or directory
java.io.IOException: Cannot run program "llvm-link": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1042)
    at shadow.Main.run(Main.java:370)
    at shadow.Main.main(Main.java:70)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:185)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
    ... 2 more

I think that OS X provides a version of llvm through XCode. Here are the details of llvm-gcc -v;

Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix

which llvm-link yields nothing though. I know OS X provides other linkers...

Any thoughts? Any additional info that I can provide that may help? Note that shadowc -n HelloWorld.shadow and shadowc -t HelloWorld.shadow both work.

[Brinsky]

It might be the case that only Clang (the LLVM-based replacement for GCC) is present on OS X, and not the entirety of the LLVM "Core" libraries and associated tools. These could presumably be built from source (which might be a painful process), but it appears that a package building system called MacPorts already offers support for it.

The MacPorts homepage provides installation instructions and a list of available "ports." LLVM 3.6 is present in the list of supported ports, and (after installing MacPorts) it should be possible to install it with sudo port -v install llvm-3.6.

I'm no expert in the workings of the LLVM toolset (or this MacPorts tool), so please be vocal with corrections.

[bwittman]

Thanks for checking out Shadow on a Mac!

It sounds like needed parts of the LLVM toolchain are missing.

How did you install LLVM? If you want to go the hardcore route, download
the source from here: http://llvm.org/releases/download.html Then use make
to compile LLVM, and then install it.

If there's something left out of your install, this hardcore installation
approach should include it. That's also a problem with the Clang Windows
binaries supplied by the LLVM project: they don't include some necessary
utilities. If the OS X LLVM calls the programs by other names, that's a
different problem.

I don't think OS X does call them by different names, however. The Darwin
binaries I looked at (
http://llvm.org/releases/3.4.2/clang+llvm-3.4.2-x86_64-apple-darwin10.9.xz)
have llvm-link.

Ultimately, gcc will link the object files on Linux, OS X, and even Windows
under mingw. However llvm-link links the LLVM bitcode, not the final
object code. In other words, it's indispensable.

On Tue, Nov 4, 2014 at 11:44 AM, Greg notifications@github.com wrote:

I've managed to install the shadow compiler on OS X using the Linux
instructions. However, when I try to compile the "Hello, World!" tutorial shadowc
HelloWorld.shadow I get the following error;

FILE DEPENDENCY ERROR: Cannot run program "llvm-link": error=2, No such file or directory
java.io.IOException: Cannot run program "llvm-link": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1042)
at shadow.Main.run(Main.java:370)
at shadow.Main.main(Main.java:70)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.(UNIXProcess.java:185)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1023)
... 2 more

I think that OS X provides a version of llvm through XCode. Here are the
details of llvm-gcc -v;

Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix

which llvm-link yields nothing though. I know OS X provides other
linkers...

Any thoughts? Any additional info that I can provide that may help? Note
that shadowc -n HelloWorld.shadow and shadowc -t HelloWorld.shadow both
work.

—
Reply to this email directly or view it on GitHub
#10.

[gdtroszak]

Yup, you were both right - OS X doesn't include all of the LLVM tools. Anyway, I installed LLVM via Homebrew (yet another OS X package manager) and now when I compile I get this error...

Assertion failed: (CLI.RetTy == Args[i].Ty && RetTys.size() == NumValues && "unexpected use of 'returned'"), function LowerCallTo, file SelectionDAGBuilder.cpp, line 7206.
0  libLLVM-3.5.dylib        0x0000000107c79bb9 llvm::sys::PrintStackTrace(__sFILE*) + 40
1  libLLVM-3.5.dylib        0x0000000107c79fa4 SignalHandler(int) + 248
2  libsystem_platform.dylib 0x00007fff93062f1a _sigtramp + 26
3  libLLVM-3.5.dylib        0x00000001081fe5ea llvm::InlineCostAnalysis::ID + 46191
4  libLLVM-3.5.dylib        0x0000000107c79e0b abort + 22
5  libLLVM-3.5.dylib        0x0000000107c79df5 abort + 0
6  libLLVM-3.5.dylib        0x0000000107c1276e llvm::SelectionDAGBuilder::visitStackmap(llvm::CallInst const&) + 0
7  libLLVM-3.5.dylib        0x0000000107c01840 llvm::SelectionDAGBuilder::LowerCallTo(llvm::ImmutableCallSite, llvm::SDValue, bool, llvm::MachineBasicBlock*) + 1290
8  libLLVM-3.5.dylib        0x0000000107bf42d9 llvm::SelectionDAGBuilder::visitCall(llvm::CallInst const&) + 1359
9  libLLVM-3.5.dylib        0x0000000107bee0ca llvm::SelectionDAGBuilder::visit(llvm::Instruction const&) + 74
10 libLLVM-3.5.dylib        0x0000000107c24ef1 llvm::SelectionDAGISel::SelectBasicBlock(llvm::ilist_iterator<llvm::Instruction const>, llvm::ilist_iterator<llvm::Instruction const>, bool&) + 33
11 libLLVM-3.5.dylib        0x0000000107c24b8c llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) + 4394
12 libLLVM-3.5.dylib        0x0000000107c22f97 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) + 847
13 libLLVM-3.5.dylib        0x0000000107d857a7 (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) + 41
14 libLLVM-3.5.dylib        0x00000001076e25ce llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 48
15 libLLVM-3.5.dylib        0x000000010787a96d llvm::FPPassManager::runOnFunction(llvm::Function&) + 297
16 libLLVM-3.5.dylib        0x000000010787ab05 llvm::FPPassManager::runOnModule(llvm::Module&) + 41
17 libLLVM-3.5.dylib        0x000000010787ae59 llvm::legacy::PassManagerImpl::run(llvm::Module&) + 645
18 llc                      0x00000001074e6684
19 libdyld.dylib            0x00007fff879b45c9 start + 1
20 libdyld.dylib            0x0000000000000004 start + 2019867196
Stack dump:
0.  Program arguments: llc -mtriple x86_64-gnu-linux -O3
1.  Running pass 'Function Pass Manager' on module '<stdin>'.
2.  Running pass 'X86 DAG->DAG Instruction Selection' on function '@main'
COMPILATION ERROR: FAILED TO COMPILE

Thoughts? Let me know if you need any more info.

[Brinsky]

Dr. Wittman can likely interpret the specific error, but I wonder if this
might be an LLVM target issue. At the moment, the compiler only
specifically targets 32-bit Windows and 64-bit Linux. While I would think
this should still work on your Mac, your llvm-gcc -v printout clearly
specifies x86_64-apple-darwin14.0.0 as your system's ideal target,
whereas the compiler (as shown in the error) is using the target x86_64-gnu-linux. I will look into altering the compiler to either
actively derive the target from the local LLVM tools, or to at least allow
manual specification in the config files.

[bwittman]

It might be the target stuff. It also might be about having assertions turned on. I remember compiling LLVM with assertions enabled and getting very similar errors.

[gdtroszak]

So I disabled LLVM assertions and now I get a LONG list of errors that have this flavor (I've just pasted the last 10 to give you a general idea);

<stdin>:82203:2: error: unknown directive
        .size   .L_string171, 40
        ^
<stdin>:82205:2: error: unknown directive
        .type   .L_array172,@object     # @_array172
        ^
<stdin>:82208:2: error: unknown directive
        .size   .L_array172, 13
        ^
<stdin>:82210:2: error: unknown directive
        .type   .L_string172,@object    # @_string172
        ^
<stdin>:82220:2: error: unknown directive
        .size   .L_string172, 40
        ^
<stdin>:82222:2: error: unknown directive
        .type   .L_array173,@object     # @_array173
        ^
<stdin>:82226:2: error: unknown directive
        .size   .L_array173, 18
        ^
<stdin>:82228:2: error: unknown directive
        .type   .L_string173,@object    # @_string173
        ^
<stdin>:82238:2: error: unknown directive
        .size   .L_string173, 40
        ^
<stdin>:82511:11: error: mach-o section specifier uses an unknown section type
        .section        ".note.GNU-stack","",@progbits

I'm still using Homebrew to install LLVM (I just used the disable-assertions option). Here is the ruby file that Homebrew uses to build it if you think it may help.

[bwittman]

We think that the compiler is using the wrong triple (x86_64-gnu-linux instead of x86_64-apple-darwin14.0.0). We're trying to fix that now, but everything takes longer than expected, and it's hard to do testing without a Mac.

[gdtroszak]

No problem or rush. Let me know if/when I can help.

[bwittman]

The latest updates should automatically detect the correct target for Mac OS. If any .meta files were generated before, please delete them before trying to recompile.

Let us know if it works!

[gdtroszak]

Nice! I'm assuming this isn't reflected in the latest release... anyway, I think I managed to build the compiler correctly (I had to skip tests to get it to work though), but I think I may be missing something.

When I run shadowc HelloWorld.shadow it spits out this message

[INFO  Configuration.java:124] No configuration files found, auto-detecting platform details
[INFO  Configuration.java:175] Non-Windows OS 'Mac OS X' detected, defaulting to Linux.ll
CONFIGURATION ERROR: Invalid path to shadow:standard: /Users/gregtroszak/Documents/workspace/shadow/target/shadow/standard

which is good except for the configuration error. After build I ran the following shell code

echo "java -jar PATH/TO/compiler-0.5.1-SNAPSHOT.jar" '$@' > /usr/local/bin/shadowc

and then tried to compile HelloWorld.shadow as usual.

I also tried just copying the jar to the same directory as HelloWorld.shadow and running

java -jar compiler-0.5.1-SHAPSHOT.jar HelloWorld.shadow

and got the same error.

These errors could (likely) be my fault, so let me know if you have any questions.

[Brinsky]

Unfortunately, proper path resolution is a painful problem. Determining where the standard libraries are relative to the compiler (or even where the compiler is) is frustratingly difficult, espcially because the testing environment is different from the release environment (the compiler is in COMPILER-DIR/target rather than just COMPILER-DIR).

You should be able to resolve this issue by moving the compiler jar up one directory out of target/. Hopefully someday this will not be necessary.

[bwittman]

This won't be an issue for people who download the compiler. For those of us working on the compiler, you can run the appropriate class files directly (easiest through Eclipse) or you can use the JAR. As was pointed out, the JAR is not built in the right place for paths to work out relative to the shadow standard library.

[gdtroszak]

Sorry for taking so long to make such a simple fix.

It seems like we're getting closer, but there is still an error. Now when I compile it spits out this...

[INFO  Configuration.java:124] No configuration files found, auto-detecting platform details
[INFO  Configuration.java:175] Non-Windows OS 'Mac OS X' detected, defaulting to Linux.ll
[INFO  Main.java:258] Generating LLVM code for CanModulus
[INFO  Main.java:258] Generating LLVM code for Class
[INFO  Main.java:258] Generating LLVM code for CanSubtract
[INFO  Main.java:258] Generating LLVM code for ArrayClass
[INFO  Main.java:258] Generating LLVM code for Number
[INFO  Main.java:258] Generating LLVM code for IllegalModificationException
[INFO  Main.java:258] Generating LLVM code for CanHash
[INFO  Main.java:258] Generating LLVM code for Reference
[INFO  Main.java:258] Generating LLVM code for UShort
[INFO  Main.java:258] Generating LLVM code for Float
[INFO  Main.java:258] Generating LLVM code for MutableString
[INFO  Main.java:258] Generating LLVM code for Enum
[INFO  Main.java:258] Generating LLVM code for BigInteger
[INFO  Main.java:258] Generating LLVM code for CastException
[INFO  Main.java:258] Generating LLVM code for Object
[INFO  Main.java:258] Generating LLVM code for HelloWorld
[INFO  Main.java:258] Generating LLVM code for CanMultiply
[INFO  Main.java:258] Generating LLVM code for Byte
[INFO  Main.java:258] Generating LLVM code for Console
[INFO  Main.java:258] Generating LLVM code for Double
[INFO  Main.java:258] Generating LLVM code for AssertException
[INFO  Main.java:258] Generating LLVM code for CanIndexStore
[INFO  Main.java:258] Generating LLVM code for File
[INFO  Main.java:258] Generating LLVM code for IOException
[INFO  Main.java:258] Generating LLVM code for Iterator
[INFO  Main.java:258] Generating LLVM code for Method
[INFO  Main.java:258] Generating LLVM code for CanCompare
[INFO  Main.java:258] Generating LLVM code for Code
[INFO  Main.java:258] Generating LLVM code for CanDivide
[INFO  Main.java:258] Generating LLVM code for CanIndex
[INFO  Main.java:258] Generating LLVM code for Long
[INFO  Main.java:258] Generating LLVM code for NumberFormatException
[INFO  Main.java:258] Generating LLVM code for ULong
[INFO  Main.java:258] Generating LLVM code for System
[INFO  Main.java:258] Generating LLVM code for IndexOutOfBoundsException
[INFO  Main.java:258] Generating LLVM code for IllegalArgumentException
[INFO  Main.java:258] Generating LLVM code for Integer
[INFO  Main.java:258] Generating LLVM code for Map
[INFO  Main.java:258] Generating LLVM code for GenericClass
[INFO  Main.java:258] Generating LLVM code for FloatingPoint
[INFO  Main.java:258] Generating LLVM code for UByte
[INFO  Main.java:258] Generating LLVM code for OutOfMemoryException
[INFO  Main.java:258] Generating LLVM code for Array
[INFO  Main.java:258] Generating LLVM code for CanAdd
[INFO  Main.java:258] Generating LLVM code for UnexpectedNullException
[INFO  Main.java:258] Generating LLVM code for CanNegate
[INFO  Main.java:258] Generating LLVM code for CanIterate
[INFO  Main.java:258] Generating LLVM code for UInt
[INFO  Main.java:258] Generating LLVM code for Int
[INFO  Main.java:258] Generating LLVM code for Exception
[INFO  Main.java:258] Generating LLVM code for UnboundMethod
[INFO  Main.java:258] Generating LLVM code for Boolean
[INFO  Main.java:258] Generating LLVM code for CanClose
[INFO  Main.java:258] Generating LLVM code for Path
[INFO  Main.java:258] Generating LLVM code for CanEqual
[INFO  Main.java:258] Generating LLVM code for CanWrite
[INFO  Main.java:258] Generating LLVM code for CanRead
[INFO  Main.java:258] Generating LLVM code for Short
[INFO  Main.java:258] Generating LLVM code for String
[INFO  Main.java:258] Generating LLVM code for AddressMap

[INFO  Main.java:160] Building for target 'x86_64-apple-darwin14'
<stdin>:2:2: error: unknown directive
        .macosx_version_min 14, 0
        ^
COMPILATION ERROR: FAILED TO ASSEMBLE