title "Pliant->C Library Wrapping Tutorial" section "foreword" header "Foreword" para [I'd like to invite any reader to comment on this tutorial or ] [ask questions relating to the topic. This is my first document like ] [this so I'm expecting there to be holes in my coverage of the topic. ] [I hope to fill these based on my own experiences and ] link "feedback" "mailto:jae@NOSPAM-zhar.net" [ from others over time. I should note that this tutorial is mainly ] [oriented toward Linux/*nix users. I'm not sure how much help it will ] [be to a Windows user. ] section "intro" header "Introduction" para [As a long time non-C hacker, I've found that a languages ability to ] [interface with C is crucial to the languages long term usefulness. ] [Particularly on my preferred platform, Linux. Though I'm sure this is ] [a truism for just about any modern platform, C (or C++) being the ] [standard systems level programming language. ] # para [One of the many things that attracted me to Pliant is the ease which ] [you can create interfaces to external C libraries. It cannot really ] [even be compared with a foreign languages interface, as its almost as ] [integrated with C as C++/ObjC. There are a few catches, but its really ] [the best I've seen in a non-C language. ] # para [Anyways... this little tutorial is meant to help you use this aspect ] [of Pliant (not as a general introduction). To date I've only partially ] [wrapped 3 C libraries, the GNU readline library, the ncurses-5 library ] [and the slang library. I'll be using the first 2 as examples as I ] [proceed, and you can download them from the ] link "resources section" "" section "resources";[.] section "basic" header "Basic Concepts" # para [Thus far I've encountered 3 basic types of C entities which need to ] [considered when wrapping a library. Functions, global variables and ] [structures. I'll first go over the basics of dealing each of these. ] # section "functions" header "Functions" # para [Wrapping function is very simple. Its simply a matter of using the ] italic [external ]; [function attribute as described on the ] link "Defining a new function" "http://pliant.cx/pliant/language/compiler/function/define.html" [ page of the documentation. Here's the basic syntax: ] listing function c_function_call argument -> result arg Type argument arg Type result external "c_library.so" "c_function" [The ]; italic [argument]; [ and ]; italic [result ]; [types ] [can either be a Pliant type or a wrapped C type. The ] italic [c_library.so ]; [library may be any cached by ld (on linux). ] [The ]; italic [ c_function ]; [is a standard function in that library. ] [Finally, you use this function in Pliant just as a standard function. ] para [The only complications in dealing with this is the types. For strings ] [or character types, special Pliant variables must be used. For strings, ] [representing 'char *' type arguments in C, use the ]; italic [ CStr ] [type. For characters, 'char' types, use the ]; italic [CChr ]; [type. ] [This is due to differences between these pliant types and ] [their C counterparts. Casting is done auto-magically (see ] link "implicit casting" "" section "implicit" [ below).] # section "globals" header "Global Variables" # para [Global variables are handled very similarly to C functions. Using the ] italic [external ]; [attribute. Again, here's the basic syntax: ] listing var Type var_name external "c_library.so" "c_global_variable" [The ]; italic [var Type var_name ]; [should seem familiar if you've ] [used Pliant at all. The extra ]; italic [external]; [ line is taken ] [from the syntax of functions, and works in the same way. The only ] [thing to watch for is to make sure the types match up. The rules ] [about character and string types mention above apply here as well. ] # section "structs" header "Structures" # para [Trickiest of these 3 are C structures. These can either be pretty ] [straight forward or a bit of a pain (comparatively, anyway). It depends ] [on whether the C compiler has done any alignment optimizations on the ] [structure or not. Its hard to give much in the way of generalities for ] [wrapping C structs, but here's what they tend to look like: ] listing type c_struct packed field Type first field Type second [The ]; italic [packed ]; [keyword is the important difference between ] [a wrapper and a normal type declaration. It keeps Pliant's compiler from ] [aligning the memory, so you can make sure it stays matched to the ] [memory layout of the wrapped structure. See the ] link "ncurses wrapper" "" section "ncurses" [ below for more an example and additional details. Particularly ] [important is the part dealing with the ] link "memory alignment" "" section "alignment"; [ problems. ] section "readline" header "Readline Wrapper" para [After briefly messing around with the interpreter, I decided that ] [the first library I had to wrap was the GNU Readline library. I'm ] [sure any other CLI lovers will empathize. So, I needed to replace ] [the interpreter's current CLI with a readline provided one. This turned ] [out to be really simple and straightforward. Its one of the things that ] [originally got me hooked. Here is everything that is required to ] [wrap the basic prompt/command line provided by the readline lib. ] listing module "/pliant/language/unsafe.pli" listing function readline prompt -> line arg CStr prompt line external "libreadline.so" "readline" [The unsafe module is needed for the automatic CStr casting. But ] [otherwise this is what most of the external function wrappers will ] [look like. To use the history features of the readline lib, the ] italic [add_history ]; [function also needs to be wrapped. ] listing function add_history line arg CStr line external "libreadline.so" "add_history" [Using these 2 functions, most of the readline's basic functionality ] [can be utilized. Here is the final function which uses both the above ] [functions to provide the new CLI functionality. ] listing function rl_get prompt -> line arg Str prompt line var CStr ret ret := readline:prompt if ret:characters = null line := "[0]" else line := ret if not line = "" and not line = "[0]" add_history:line [As you can see, the two wrapped function work just as if they were ] [Pliant functions. The '[lb]0[rb]' is returned in place of a null, as the ] [interpreter is expecting a normal Pliant string and Pliant Str types ] [can't hold nulls except using their format, ie. '[lb]0[rb]'. ] section "ncurses" header "Ncurses-5 Wrapper" # para [The Ncurses wrapper is more complicated than the readline wrapper. As ] [it not only has many functions to wrap, it has global variables and ] [a C structure to wrap. ] # section "nc-funs" header "Function Wrapping" # para [Wrapping a function for Ncurses is much the same as for the readline ] [library, just a lot more of them. So there isn't much new to go into ] [in this section. Ncurses does have one twist when compared to ] [readline, that is that it has quite a few functions exported as ] [macros. In C, these are incorporated into the program by the ] [compiler. This doesn't help, since we're accessing the library ] [directly. ] para [To deal with macros, you need to port them over manually. This is ] [really pretty simple, as macros are defined in the header file and ] [are usually constructed from the other functions exported from the ] [library or an extremely simple bit of code. Each is handled in ] [basically the same way. You simply re-implement them in Pliant. ] [The only difference is whether you use another function exported ] [by the library or not. ] para [Here's an example of a reimplemented macro from the Ncurses wrapper. ] [First, the C: ] listing #define touchwin(win) wtouchln((win), 0, getmaxy(win), 1) [And the Pliant version: ] listing function touchwin w arg Address w wtouchln w 0 (w:_maxy + 1) 1 # para [The ]; italic [touchwin ]; [function in ncurses marks a window ] [so that the next refresh redraws it. It is implemented in C via ] [a macro calling the ]; italic [wtouchln]; [ function (which marks ] [a range of 1 or more lines for redraw) on the contents of the ] [target window. The Pliant version does the same. The only ] [is that instead of calling ]; italic [getmaxy ]; [(which is yet ] [another macro), it accesses the window structure to get the number ] [of lines in the window (ie. ]; italic [_maxy];[). ] # section "nc-vars" header "Global Variable Wrapping" # para [Global variables are easy to wrap and there's not much to complicate ] [things. So let's get right to an example from Ncurses: ] listing var Address stdscr external curses "stdscr" para [In Ncurses ]; italic [stdscr ]; [is the address of the default window ] [created when you initialize the display. Pretty simple... eh. ] # section "nc-structs" header "C Structure Wrapping" # para [The most difficult part of writing my Ncurses wrapper was dealing ] [with Ncurses window structure. It probably wouldn't have been as ] [hard for someone with more C experience than I, but with help from the ] link "forum" "http://pliant.cams.ehess.fr:8080/pliant/forum.html" [ I figured it out. ] para [There are several steps I went through to get a working wrapper that ] [I was happy with, and this section will reflect my development path. ] [To start we need to know what we're dealing with.] section "c_struct" header "The C structure" # listing struct _win_st { short _cury, _curx; /* current cursor position */ /* window location and size */ short _maxy, _maxx; /* maximums of x and y, NOT window size */ short _begy, _begx; /* screen coords of upper-left-hand corner */ short _flags; /* window state flags */ /* attribute tracking */ attr_t _attrs; /* current attribute for non-space character */ chtype _bkgd; /* current background char/attribute pair */ listing /* option values set by user */ bool _notimeout; /* no time out on function-key entry? */ bool _clear; /* consider all data in the window invalid? */ bool _leaveok; /* OK to not reset cursor on exit? */ bool _scroll; /* OK to scroll this window? */ bool _idlok; /* OK to use insert/delete line? */ bool _idcok; /* OK to use insert/delete char? */ bool _immed; /* window in immed mode? (not yet used) */ bool _sync; /* window in sync mode? */ bool _use_keypad; /* process function keys into KEY_ symbols? */ int _delay; /* 0 = nodelay, <0 = blocking, >0 = delay */ listing struct ldat *_line; /* the actual line data */ listing /* global screen state */ short _regtop; /* top line of scrolling region */ short _regbottom; /* bottom line of scrolling region */ /* these are used only if this is a sub-window */ int _parx; /* x coordinate of this window in parent */ int _pary; /* y coordinate of this window in parent */ WINDOW *_parent; /* pointer to parent if a sub-window */ /* these are used only if this is a pad */ struct pdat { short _pad_y, _pad_x; short _pad_top, _pad_left; short _pad_bottom, _pad_right; } _pad; short _yoffset; /* real begy is _begy + _yoffset */ }; # para [I won't get into the gory details of the above, but there are a ] [items which need more explanation for what comes next to make sense. ] [First, it should be noted that the ]; italic [bool ]; [type used ] [above is an unsigned char and the ]; italic [attr_t ]; [type is an ] [unsigned long. Finally there are the two other structures used] [it this struct, namely ]; italic [ldat ]; [and ]; italic [pdat]; [. The latter is already defined, so here's the C for the former: ] listing struct ldat { chtype *text; /* text of the line */ short firstchar; /* first changed character in the line */ short lastchar; /* last changed character in the line */ short oldindex; /* index of the line at last update */ }; para italic [chtype ]; [is an unsigned long int.]; eol [As you will see below, I don't actually wrap this struct. Ncurses ] [provides functions to deal with this struct via the window. So all ] [that will be needed is to store the address in the window struct. ] section "struct_wrap" header "The Pliant version" para [First, lets wrap that ]; italic [pdat ]; [struct: ] listing type PDat packed field Int16 _pad_y _pad_x field Int16 _pad_top _pad_left field Int16 _pad_bottom _pad_right para [This creates the new type in Pliant, ]; italic [PDat]; [. It looks ] [pretty much just like the example, and there are no real surprises ] [here. Notice here and below that the choice as to which Pliant type ] [to use in the wrapper is based primarily on the size of the type ] [its mapping. That's why you'll see the ]; italic [bool ]; [type, ] [an unsigned char, in the C struct replaced with ]; italic [uInt8] [, an 8 bit Int (the same size as the wrapped char type).] listing public type Window packed listing field Int16 _cury _curx # current cursor position # window location and size field Int16 _maxy _maxx # maximums of x and y, NOT window size field Int16 _begy _begx # screen coords of upper-left-hand corner field Int16 _flags # window state flags # padding to correct offset bold listing field (Array Byte 2) padding1 listing # attribute tracking field uInt32 _attrs # current attribute for non-space character field uInt32 _bkgd # current background char/attribute pair listing # option values set by user field uInt8 _notimeout # no time out on function-key entry? field uInt8 _clear # consider all data in the window invalid? field uInt8 _leaveok # OK to not reset cursor on exit? field uInt8 _scroll # OK to scroll this window? field uInt8 _idlok # OK to use insert/delete line? field uInt8 _idcok # OK to use insert/delete char? field uInt8 _immed # window in immed mode? (not yet used) field uInt8 _sync # window in sync mode? field uInt8 _use_keypad # process function keys into KEY_ symbols? # padding to correct offset bold listing field (Array Byte 3) padding2 listing field Int _delay # 0 = nodelay, <0 = blocking, >0 = delay listing field Address _line # the actual line data listing # global screen state field Int16 _regtop # top line of scrolling region field Int16 _regbottom # bottom line of scrolling region # these are used only if this is a sub-window field Int _parx # x coordinate of this window in parent field Int _pary # y coordinate of this window in parent field Address _parent # pointer to parent if a sub-window # these are used only if this is a pad field PDat _pad field Int16 _yoffset # real begy is _begy + _yoffset # padding to correct offset - not really needed, as its at the end bold listing field (Array Byte 2) padding3 para [Most of this is pretty straight forward, expept those lines I've ] [highlighted in ]; bold [bold]; [. ] [These are present due to the C compiler aligning ] [the memory of the structure as an optimization trick. How you ] [figure out if you need these and how much padding to add is ] [covered next. ] # section "alignment" header "Memory Alignment with C structures" # para [There are 2 related problems to solve if the struct you're writing ] [gets aligned by the C compiler. First is determining this and then ] [what to do about it. ] para [If you think you've got your wrapper right, yet it still won't ] [map onto the C struct its supposed to be wrapping (basically if ] [all else fails), it's probably due to alignment issues. To be ] [sure, compare the sizes of the C struct vs. the Pliant type. ] para [In Pliant this is a snap, as all types have the built-in method ] italic [size ]; [which, oddly enough, returns the memory size of the ] [type. So, to check the size of the above Pliant Window type, in ] [the file that contains its source I'd just add: ] listing console "Window size: " (Window size) eol para [In case you're curious, the correct size is 76. ] para [To determine the size of the struct in C, you need to whip up a ] [little C program. Here is the program I wrote to find out the ] [size of the ]; italic [WINDOW ]; [struct Ncurses defines: ] listing #include #include int main() { WINDOW w; printf("linesize: %d\n",sizeof(w)); } para [After running this, you can see for sure whether the memory sizes ] [of the Pliant and C code are the same. If the Pliant's type size ] [is larger than the C struct, you need to check the field types you ] [used in the Pliant type (eg. an uInt instead of uInt16). If the C ] [struct is the larger of the two, then you probably need to add ] [padding to the Pliant type. How much and where? To determine this ] [you need to determine the offsets used for each variable in the ] [struct. Here's the abridged version of the program I used to determine ] [this for this: ] listing #include #include int main() { WINDOW w; printf("cury offset: %d\n",offsetof(struct _win_st,_cury)); printf("curx offset: %d\n",offsetof(struct _win_st,_curx)); [... cut-n-paste code snipped ...] printf("pad offset: %d\n",offsetof(struct _win_st,_pad)); printf("yoffset offset: %d\n",offsetof(struct _win_st,_yoffset)); } # para [After you have this information, you compare each offset with the ] [size of the appropriate variable and look for increases in the ] [offset which are different from the variable size. If so, you've ] [found a place where padding needs to be added. I use arrays of ] [bytes to explicitally allocate these chunks (Hubert suggested this). ] # section "implicit" header "The Magic of Implicit Casting" # para [Now that we have our wrapper, there's one last detail to make it ] [easy to deal with... ] para [In Ncurses, all the functions and variables deal with the Window ] [as an Address. They expect the windows passed in as addresses and ] [return addresses. While you also need the information contained in ] [the struct (wrapped via the pliant type) for things like the macro ] [replacement above. It is possible to manually cast these each time, ] [but that is a pain. Instead it is possible to have them cast ] [automatically when necessary via implicit casting. ] para [Setting up implicit casting for automatically converting Address types ] [to Window types and vice versa requires 2 separate casting functions. ] eol; [First, casting from a Window to an Address: ] listing function 'cast Address' w -> a arg Window w ; arg Address a implicit a := addressof:w listing export 'cast Address para [Now, the other way: ] listing function 'cast Window' a -> w arg Window w ; arg Address a implicit w := (a map Window) listing export 'cast Window' para [For more examples of implicit casting, see ] link "cchar.pli" "http://pliant.cx/pliant/browse/file/pliant/language/type/text/cchar.pli" [ and ] link "cstr.pli" "http://pliant.cx/pliant/browse/file/pliant/language/type/text/cstr.pli" [ in the Pliant distibution. ] # section "end" header "Questions?" para [Some parts of this unclear? Have an idea for an improvement? ] # section "resources" header "Resources" para [I want to do a little more work on the Ncurses wrapper before releasing ] [it. But here's a link to the readline wrapper, such as it is. ] eol; eol; list item link "readline.pli" "http://zhar.net/projects/pliant/samples/readline/download/" [ - the simple wrapper for gnu's readline library] item [plncurses.tar.gz - will be a collection of modules for ncurses ] [wrapping]