[Fonc] Compile FONC on OSX 10.12?

[Eric Gade]

Hello,

I've only recently discovered VPRI and FONC. I spent a couple of nights
this week trying to compile the source I obtained from this SVN:

`svn checkout http://piumarta.com/svn2/idst/tags/idst-376 fonc-stable`

My first attempt to run `make` without any custom flags/configuration
resulted in the following error:

/bin/sh -ec 'ln -s ../gc6.7/gc.a .'
cc -Iinclude -DPREFIX='"/usr/local/lib/idc/i386-apple-darwin16.1.0/"' -g
-Wall -Wreturn-type -Werror -fno-common -arch i486 -DNDEBUG
-DSYSARCH=\"i386\" -DSYSOS=\"darwin\" -O3 -march=prescott
-fomit-frame-pointer -falign-functions=16 -funroll-loops -c src/libid.c
-o libid.o
clang: error: optimization flag '-falign-functions=16' is not supported
make[2]: *** [libid.o] Error 1
make[1]: [boot] Error 2 (ignored)
/bin/sh -ec '[ -d stage1 ] || mkdir stage1'
/bin/sh -ec 'for dir in id st80 idc; do ( echo $dir; cd $dir;
/Applications/Xcode.app/Contents/Developer/usr/bin/make IDC="../boot/idc
-B../boot/ -O" BIN="../stage1/" ); done'
id
make[2]: Nothing to be done for `all'.
st80
../boot/idc -B../boot/ -O -k -c _object.st -o ../stage1/_object.o
../boot/idc: line 242: ../boot/idc1: No such file or directory
make[2]: *** [../stage1/_object.o] Error 127
make[1]: *** [stage1] Error 2
make: *** [all] Error 2

So I cleaned and then ran `make config` and forced the flags to not include
`-falign-functions=16`, cleaned, like so:

`make config CC="g++" O3FLAGS="-O3 -march=prescott -fomit-frame-pointer
-funroll-loops" PGFLAGS="-O3 -march=prescott -funroll-loops"`

Followed by `make`, which threw tons of errors like this:

src/libid.c:894:3: error: array index -1 is before the beginning of the
array [-Werror,-Warray-bounds]
_vtable_vtable->_vtable[-1]= _vtable_vtable;
^ ~~
src/libid.c:154:5: note: array '_vtable' declared here
oop _vtable[0];
^
src/libid.c:897:3: error: array index -1 is before the beginning of the
array [-Werror,-Warray-bounds]
_vtable_vtable->vtable.bindings->_vtable[-1]= _vector_vtable;
^ ~~
src/libid.c:154:5: note: array '_vtable' declared here
oop _vtable[0];
^
src/libid.c:898:3: error: array index -1 is before the beginning of the
array [-Werror,-Warray-bounds]
_vector_vtable->vtable.bindings->_vtable[-1]= _vector_vtable;
^ ~~
src/libid.c:154:5: note: array '_vtable' declared here
oop _vtable[0];
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.


I'll admit that my experience with C is rudimentary, which might mean I
should even try to be playing with this stuff. But oh boy, do I want to
play wth it.

If this is not the latest version of the code, or if these projects are now
being supported elsewhere, please let me know where to look. Thanks!

[Renaud Hébert]

I think that your first fix is good: apparently the makefile use gcc
specific option, so CC="g++" as you did should fix this.

For your second issue, it is likely that it is a wrong check that the index
array is positive but a classic trick in C is to put things before the
array.
Normally this check is only a warning but here there is the option -Werror
which makes all warning an error so the build fail..
You could
1) remove the array-bound check adding an option -Wno-array-bounds
2) remove the option -Werror


I hope this helps.

Post by Eric Gade
Hello,
I've only recently discovered VPRI and FONC. I spent a couple of nights
`svn checkout http://piumarta.com/svn2/idst/tags/idst-376 fonc-stable`
My first attempt to run `make` without any custom flags/configuration
/bin/sh -ec 'ln -s ../gc6.7/gc.a .'
cc -Iinclude -DPREFIX='"/usr/local/lib/idc/i386-apple-darwin16.1.0/"' -g
-Wall -Wreturn-type -Werror -fno-common -arch i486 -DNDEBUG
-DSYSARCH=\"i386\" -DSYSOS=\"darwin\" -O3 -march=prescott
-fomit-frame-pointer -falign-functions=16 -funroll-loops -c src/libid.c
-o libid.o
clang: error: optimization flag '-falign-functions=16' is not supported
make[2]: *** [libid.o] Error 1
make[1]: [boot] Error 2 (ignored)
/bin/sh -ec '[ -d stage1 ] || mkdir stage1'
/bin/sh -ec 'for dir in id st80 idc; do ( echo $dir; cd $dir;
/Applications/Xcode.app/Contents/Developer/usr/bin/make IDC="../boot/idc
-B../boot/ -O" BIN="../stage1/" ); done'
id
make[2]: Nothing to be done for `all'.
st80
../boot/idc -B../boot/ -O -k -c _object.st -o ../stage1/_object.o
../boot/idc: line 242: ../boot/idc1: No such file or directory
make[2]: *** [../stage1/_object.o] Error 127
make[1]: *** [stage1] Error 2
make: *** [all] Error 2
So I cleaned and then ran `make config` and forced the flags to not
`make config CC="g++" O3FLAGS="-O3 -march=prescott -fomit-frame-pointer
-funroll-loops" PGFLAGS="-O3 -march=prescott -funroll-loops"`
src/libid.c:894:3: error: array index -1 is before the beginning of the
array [-Werror,-Warray-bounds]
_vtable_vtable->_vtable[-1]= _vtable_vtable;
^ ~~
src/libid.c:154:5: note: array '_vtable' declared here
oop _vtable[0];
^
src/libid.c:897:3: error: array index -1 is before the beginning of the
array [-Werror,-Warray-bounds]
_vtable_vtable->vtable.bindings->_vtable[-1]= _vector_vtable;
^ ~~
src/libid.c:154:5: note: array '_vtable' declared here
oop _vtable[0];
^
src/libid.c:898:3: error: array index -1 is before the beginning of the
array [-Werror,-Warray-bounds]
_vector_vtable->vtable.bindings->_vtable[-1]= _vector_vtable;
^ ~~
src/libid.c:154:5: note: array '_vtable' declared here
oop _vtable[0];
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
I'll admit that my experience with C is rudimentary, which might mean I
should even try to be playing with this stuff. But oh boy, do I want to
play wth it.
If this is not the latest version of the code, or if these projects are
now being supported elsewhere, please let me know where to look. Thanks!
--
Eric
_______________________________________________
Fonc mailing list
http://mailman.vpri.org/mailman/listinfo/fonc_mailman.vpri.org

[Eric Gade]

Hi Tim,

Thanks for all this. It's a really good start. However it seems like we
might not be working on the same codebase, which might have something to do
with where the latest stable source is hosted or something.

You asked:

*"Is there a reason you are trying to build from an older version? The

version in trunk has been stable for many years, and is the version I’ve
been using for quite some time."*

I used that specific URL because that's the latest "stable" version
according to this Wiki, which must be out of date:
http://vpri.org/fonc_wiki/index.php?title=Installation_Guide

Either way, when I grabbed code svn told me it had checked out "revision
650". If I instead grab the source from the trunk, it says the same, and
the code appears to be identical (I'm getting trunk from the "Official
Development Version" link in that same Wiki).

This might be the cause of some of the trouble I had with your later
instructions, which seemed to involve code that is different from what I'm
seeing (more on this in a bit).

Aside: is there any reason -- legal or political or preference, whatever --
that the latest stable version of this code can't be hosted on Github? I'm
asking because it's quite easy to comment on code using that platform and
it would be very clear what the latest versions are. I'm willing to do this
if others are open to it, though obviously I cannot maintain code that I
don't understand or haven't worked on.

As for the `configure` file instructions -- nice! That was very helpful,
and seems to have resolved some of the other errors. However, now if I run
`make config` then `make` in the root folder, it throws this error:

```
st80
make[2]: *** No rule to make target `../stage1/_object.o', needed by
`../stage1/st80.so'. Stop.
make[1]: *** [stage1] Error 2
make: *** [all] Error 2
```

So perhaps something is missing.

This could very well be because I installed the gc (version 7.6) via
homebrew, since I had totally separate nastiness trying to compile it
myself. Not sure.

As for the later instructions re: jolt, I made the header and c file for
the allocator as you instructed (thanks), but there are two issues related
to differences in the code we are seeing:

First, in `CodeGenerator-i386`, `DynamicIntel32CodeGenerator` does not have
a mesage/method called `generate: tree` at all, but instead has `emit:
tree`, which looks like this:

```
DynamicIntel32CodeGenerator emit: tree
[
| _entry |
{ asm_pass= 0; }.
{ asm_pc= 0; }.
self finaliseFrame.
tree emit: self.
CompilerOptions verboseExec ifTrue: [{ printf("code size %d bytes\n",
(int)asm_pc); }].
{ asm_pc= (insn *)malloc((long)asm_pc); }.
{ asm_pass= 2; }.
{ v__entry= (oop)asm_pc; }.
self relocateLabels_: self _asmPC.
tree emit: self.
CompilerOptions verboseExec ifTrue: [{ printf("code start %p\n", (void
*)v__entry); }].
]
```
Similar enough to what you described, but I don't want to screw this up.

Second, `Compiler.st` doesn't have a `Label free` block at all.

I think we are getting closer, so thanks again for the help (and from the
others who responded too!). I will assemble a README for getting this to
work in Sierra (OSX 10.12) if/when I can get it all running.

Post by Eric Gade
Hello,
I've only recently discovered VPRI and FONC. I spent a couple of nights
`svn checkout http://piumarta.com/svn2/idst/tags/idst-376 fonc-stable`

I’ve been playing with idst on MacOS for a number of years; there are a
lot of interesting things to explore in it. I’m currently running it on
Mac OS 10.9.5 (darwin13).
Is there a reason you are trying to build from an older version? The
version in trunk has been stable for many years, and is the version I’ve
been using for quite some time. Anyway, here are the changes I’ve made to
1) get the latest Bohm garbage collector: https://www.hboehm.info/gc/gc_
source/ (I’m using 7.0)
2) Figure out which Darwin kernel version you are running (I’m on
< darwin13 () {
< CC=${CC:-"cc"}
< PREFIX=${PREFIX:-"/usr/local/lib/idc/$TARGET/"}
< CFLAGS=${CFLAGS:-"-g -Wall -Wreturn-type -Wno-array-bounds
-fno-common -fwrapv"}
< OBJEXT_O=${OBJEXT_O:-".o"}
< CCFLAGS_O=${CCFLAGS_O:-"-c"}
< LDFLAGS_SO=${LDFLAGS_SO:-"-bundle -flat_namespace -undefined
suppress"}
< OBJEXT_SO=${OBJEXT_SO:-".so"}
< SYSOS="darwin"
< }
and put a reference to that in the $TARGET case selector in the configure
i[3456789]86-*-darwin13*) i386; darwin13; arch_386; gc7 ;;
< LDLIBS=${LDLIBS:-"-ldl -lm -lpthread”}
I think that’s all you might need to change to get idc going. To get Jolt
to run dynamically-generated x86 code, you will need to change the memory
allocation for code sections to use mmap and mprotect, rather than malloc.
==== alloc-code.h ====
#ifndef __alloc_code_h
#define __alloc_code_h
#include <sys/types.h>
#include <sys/mman.h>
extern void* alloc_code(size_t size);
extern void free_code(void *addr);
extern void make_executable(void *addr, size_t size);
#endif
==== alloc-code.c ====
#include "alloc-code.h"
#include <stdio.h>
#include <stdlib.h>
void *
alloc_code(size_t size)
{
long *addr;
addr = mmap(NULL, size+4, PROT_READ | PROT_WRITE, MAP_ANON |
MAP_PRIVATE, -1, 0);
if (addr == MAP_FAILED) {
fprintf(stderr, "mmap failed allocating %zu bytes\n", size);
exit(1);
}
*addr = size;
return addr + 1;
}
void
free_code(void *addr)
{
int error;
long *laddr = (long *)addr - 1;
error = munmap(laddr, *laddr);
if (error) {
fprintf(stderr, "munmap failed for address %ld\n", (long)laddr);
exit(1);
}
}
void
make_executable(void *addr, size_t size)
{
int error;
long *laddr = (long *)addr - 1;
error = mprotect(laddr, size, PROT_READ | PROT_EXEC);
if (error) {
fprintf(stderr, "mprotect failed for address %ld\n", (long)addr);
exit(1);
}
}
====
And change the CodeGenerator-i386 files and Compiler.st to use the
- in CodeGenerator-i386.st, in DynamicIntel32CodeGenerator generate: tree
method, change to use the alloc_code method, and call make_executable after
DynamicIntel32CodeGenerator generate: tree
[
| _entry _size |
{ asm_pass= 0; }.
{ asm_pc= 0; }.
self finaliseFrame.
tree generate: self.
CompilerOptions verboseExec ifTrue: [{ printf("code size %d bytes\n",
(int)asm_pc); }].
{ v__size= (oop)asm_pc; }.
{ asm_pc= (insn *)alloc_code((size_t)asm_pc); }.
{ asm_pass= 2; }.
{ v__entry= (oop)asm_pc; }.
self relocateLabels_: self _asmPC.
tree generate: self.
CompilerOptions verboseExec ifTrue: [{ printf("code start %p\n", (void
*)v__entry); }].
{ iflush((void *)v__entry, (void *)asm_pc); }.
{ make_executable ((void *)v__entry, (size_t)v__size); }.
" CompilerOptions verboseExec ifTrue: [{ disassemble((void *)v__entry,
(void *)asm_pc); }]."
^Array with: _entry with: _size.
{ include "alloc-code.h" }
Label free
[
CompilerOptions verboseExec ifTrue: [{ printf("free %p\n",
self->v__address); }].
{ free_code((void *)self->v__address); }.
]
Good luck, and happy exploring!
— tim

[Eric Gade]

Definitely making progress, but not quite there I don't think. Now I'm
getting a segfault:

```
cc -g -Wall -Wreturn-type -Wno-array-bounds -fno-common -fwrapv -arch i486
-g -I../stage1//include -DNDEBUG -DSYSARCH=\"i386\"
-DSYSOS=\"darwin\" -fno-strict-aliasing -g
-DPREFIX='"/usr/local/lib/idc/i386-apple-darwin16.1.0/"' libid.c -c -o
../stage1/libid_g.o
/bin/sh -ec '( cd st80;
/Applications/Xcode.app/Contents/Developer/usr/bin/make IDC="../boot/idc
-B../boot/ -O" BIN="../stage1/" )'
../boot/idc -B../boot/ -O -k -c _object.st -o ../stage1/_object.o
../boot/idc: line 300: 80920 Segmentation fault: 11 $run ${prefix}idc1
-I$prefix $idopts $idflags $infile -o $ccfile
make[2]: *** [../stage1/_object.o] Error 139
make[1]: *** [stage1/st80.so] Error 2
make: *** [all] Error 2
```

By the way I get this both when I try to build for `i386` AND `x86_64`. The
reason I'm trying both is because I have a newer MacBook and, well, I'm
just not sure which one to use. I've learned that Apple has muddied the
waters here. For example:
Running `arch` gives you `i386`
But running `uname -m` gives you `x86_64`.

Now the segfault there might be related to not having these parts
configured properly, but I tried every combination I could think of that
made sense when editing `object/boot/configuration` and I always get this
segfault.

Ok, it looks like the changes required are limited to the boot/configure
file and a simple change in gc-7.0/mach_dep.c
https://drive.google.com/open?id=0B4_HuTz-3Kw0Y3hfTTMxZUZEQkk
cd object/boot
patch < configure.diff.txt
cd object/gc-7.0
patch < mach_dep.c.diff.txt
— tim