Webio programmers manual

Introduction

Webio includes:

Internal structure

Webio is written entirely in C language, and has been tested on Microsoft and GCC compilers. Most source files are compiled on all target systems with no "ifdefs or modifications - we've tried to limit those to a few selected include files.

These source and header files may need to be modified or replaced when porting Webio./span>

	websys.c	Windows vs. Linux "ifdef" code
	websys.h	System definitions, may include windowsdefs.h or linuxdefs.h
	windowsdefs.h	Windows port definitions
	linuxdefs.h	Linux port definitions

wi_poll() it designed to be driven by a single thread. The routine the routine wi_thread(); , is provided to implement this. easily. The demo program demonstrates how it to this. It's sole thread, the main() routine sets up the network (ifdeffed for Linux or windows) and then blocks in a call to wi_thread(); . When this call returns, the application has terminated.

All the objects keep stateful information, so if the execution is required to block, the code can simply return to the main thread, or whatever is calling wi_poll()

wi_sess
One of these objects is created for every connection which is opened to the web server. Is contains the open socket, buffers for reading and writing, and pointers to any related file objects.

The main poll activity of the program works largely by traversing a linked list of wi_sess; objects and checking each for any pending work which can now be done because of changes since the last poll.

wi_file
This is a wrapper for a lower layer FILE descriptor. One of these is maintained by the server for each open file. The lower layer descriptor may be a conventional disk file, one of the embedded files produced by fsbuilder, or other types of files defined by the programmer.

wi_filesys
One of these objects is created for each type of file supported in the build. Typically there will be one foe disk files and one for fsbuilder embeddded files.

The object is primarily a list of routines which perform common ansi-like file operations, such as fopen, fread, and fwrite. New file systems may be implemented by defining another one of these objects and provided it's member functions.

We deal with the by allocating and freeing all our memory via the macros WI_MALLOC() and WI_FREE() .

The demo application

Building on Linux with GCC

• un-tar or otherwise download the distribution.
• In needed, build the fsbuilder utility ("make")
• build the server with "make -f Makefile"
• Start the server (must be root): "./webio &"

Building on Windows via Visual Studio

• un-zip or otherwise download the distribution.
• With Visual studio v7.0 (or newer) open the Webio.dsw project file
• Click the build button. Check the output window for errors.
• Run the server: "Debug/webio"

Merging Webio with your target or application

Initialization

1) Start the network (not needed on Linux)

On at least some versions of windows, you will need to start the "winsock" network services by calling WSAStartup(), as in the webtest.c example. Most Unix-like systems don't require this; their networks are already running when your application loads. Other OS's may have needs like Windows - check your OS networking documents to be sure.

2) call wi_init() to Initialize the web server.

This sets up Webio internals and preforms the sockets listen() call which connects Webio to the network.

1) The counting timer, cticks.

This is defined as an unsigned long which is incremented by one at regular intervals, ideally about 1/10 of a second. The number of "ticks" per second is given by the #define TPS. It is the responsibility of the porting engineer to make sure that cticks in incremented at the TPS rate. Neglecting to do this will disable the web servers ability to time out idle connection.

2) CPU cycles (i.e. a thread).

The File system builder, fsbuilder

• Ram based files allow much faster access than disk files.
• Prevents users from modifying sensitive GUI files on the disk.
• Allow operation on systems without a conventional file system.
• Allow a tight coupling of dynamic page content and C routines.
• Reduce the number of files in the installation package

Once built, fsbuilder is meant to be run as a command line application on the development system. The Webio Linux demo makefiles will invoke it as needed to produce new C file whenever the content changes. fsbuilder is NOT meant to be part of the program containing Webio .

fsbuilder takes a list of files as it's only required command line argument. The "listfile" is a plain text file, with one file name per line. Generally, static content files such as text and images which are in the list should exist on the working directory (or in an indicated path). The list may also include names of SSI and CGI files, for which placeholders are created by fsbuilder.

The listfile may contain comment lines starting with "#";

In addition to these per-file options, the following command line options will efect ALL files:

	-a	Requests for the file will require authentication
	-c	Enable cache control - Browser will not cache the file
	-o	send C data array output to named file
	-h	send C headers to named file

Dynamic files

dynamic content

This section describes how to use Webio's two mains types of dynamic content, SSI and CGI. Both are implemented as "pseudo" files - entries in the em_file list which do not have any array of static data associated with them. When the core server loop does a read on these files, rather than copying out data the embedded file system code calls a C routine, passing the wi_sess structure ( aka a session) to the C routine. The C routine fills in a per-session buffer with the HTML (or other content) which is to be returned to the browser.

The exact semantics of the C routines which service SSI and CGI functions are different, but in both cases a stub routine is generated by the fsbuilder utility and left (by default) in the file wsfdata.c. This file should not be compiled directly, rather the stubs it contains should be copied to a user maintained file, which IS compiled and linked with the webio system. This allows you to add your own code to the stubs without it being overwritten the next time you build.

Lets look at the details of SSI via an example, one which places the current time in an HTML page. The HTML might look like this:

<!--#include file="time_of_day.ssi" -->

time_of_day.ssi -s time_of_day_routine

The C stub would look like:

/* time_of_day_routine()
 *
 * SSI routine stub
 */

int
time_of_day_routine(wi_sess * sess, EOFILE * eofile)
{
   /* Add your code here */
   return 0;
}

Lastly, you need to provide code which puts the text/html to be displayed in the output buffer passed in sess. The buffer has it's own data type, txbuf, which has a fairly complex semantics because of the issues with allocating buffers as we need to write to them, and freeing them as they are written to the TCP socket. Fortunately you don;t need to deal with this - webio's library includes the function wi_printf(), which outputs formatted text into the session's output buffer.

Using wi_printf(), your time of day function might look something like this:

/* time_of_day_routine()
 *
 * Sample SII routine to print TOD into an output html file. 
 *
 */

int 
time_of_day_routine(wi_sess * sess, EOFILE * eofile)
{
   char * time_String; 

   time_String = get_system_time(); /* call OS for current time */

   wi_printf(sess, "Time is now: %s <br>", time_String );

   return 0;      /* OK return code */
}

Since the original include statement was in an HTML file, HTML can be included in the text written to the output buffer. In this case, a line break was inserted at end of line. Controlling HTML generation on-the-fly with C code can be a very powerful tool. Javascript can also be created by SSI routines.

To understand how this works, let examine it's use in the demo server. The demo creates an SSI file which display the number of total memory blocks allocated by the server since it's been running. The listfile contains this line:

memhits.var -e u_long  wi_totalblocks

...and mem cexp: <!--#include file="memhits.var" -->

...and mem cexp: 51243

A single step is required bny ther programmer to support all the C-code expressions in the system - you must copy the routine wi_cvariables() from the wsfcode.c file generated by fsbuilder into a C file which is linked into the system. As with other SSI and CGI code, the generated code is created in a "non live" file so that you can modify it as needed, and future runs of fsbuilder will not overwrite it.

The advantages of using the C-code expressions "-e" option over the SSI "-s" option are less work during development, and less overhead during runtime.

WARNING
When fsbuilder generates the code for wi_cvariables(), is generates a switch: statement with algorithmically generated case: labels. These labels include several digits, based on the order of the files in the listfile, to ensure uniqueness. For example in the demo example the lable might be "MEMHITS_VAR5" if the file is the fifth in the listfile.

The "gotcha" this creates is when you add more files to the listfile ahead of the "-e" file. This changes the label (in this case from "MEMHITS_VAR5" to something like "MEMHITS_VAR8") in both the ".c" and the ".h" output files. Since the the header (".h") file are included in the compiled code and the ".c" files are not, this creates a discrepancy between name of the label in the two files, and at compile time you will get an error something like:

gcc -g -DLINUX -c webtest.c -o webtest.o
webtest.c: In function "wi_cvariables":
webtest.c:157: error: "MEMHITS_VAR5" undeclared (first use in this function)

CGI

# form handlers
testaction.cgi -f testaction_cgi

/* testaction_cgi
 * Stub routine for form processing
 * 
 * Returns NULL if OK, else short error text
 */

char *
testaction_cgi(wi_sess * sess, EOFILE * eofile)
{
   char *   your_name;

   your_name = wi_formvalue(sess, "your_name");   /* default: John */


   /* Add your code here */

   return 0;
}

In the event that Webio can't parse the HTML (hey, it happens) it may be useful to explain how the form's name-value info is passed to the CGI routine. Whenever a HTTP GET or POST request is received, the attached name-value text is parsed into a wi_form object. The definition of the object is in the webio.h file. The session objects's (wi_sess) ws_formlist field is set to point to this form structure.

As the name implies, the forms can be a linked list, however there is usually only one. Multiple forms may be attached to a session if a page invoked with name/value pairs includes and SSI which also has name/vale pairs.

Form structures are automatically freed by the Webio internals when the page processing is complete. Webio also provides the utility function wi_formvalue() to return the text for a value in the form when the corresponding name is passed.

After the form name/value pairs have been processed, the browser expects a page to be displayed in response to the GET or POST request. Since the file name in the request was directed to the CGI routine, there is no actual file associated with the request. If would be possible for the C routine to generate the entire file (as conventional CGI systems often do), however that can mean a lot of tedious extra code. To avoid this, use the wi_redirect() function described ion the next section.

Compile-time options

/*********** Optional webio features - comment out to remove ***************/

#define WI_STDFILES  1     /* Allow host system (Linux, Windows) files */
#define WI_EMBFILES  1     /* Allow embedded FS files */

#define WI_THREAD    1     /* Drive webio with a thread rather than polling */


/*********** Webio sizes and limits ***************/

#define WI_RXBUFSIZE    1536  /* rxbuf[] total size */
#define WI_TXBUFSIZE    1400  /* txbuf[] section size */
#define WI_MAXURLSIZE   512   /* URL buffer size  */
#define WI_FSBUFSIZE    4096  /* file read buffer size */

#define WI_PERSISTTMO   300   /* persistent connection timeout */

authentication

int         file_auth_routine(void * fd, char * name, char * pw);  /* Optional, for authentication */

This is most efficient when used with embedded files. Each line in the listfile which contains the option "-a" will cause fsbuilder will set a flag in the embedded file's descriptor. This flag can be used to allow universal access to non-flagged files and restrict access to flagged files. Note that the programmer must still provide a routine (and name/password database) to resolve access. An example of this is in the wfs_auth() routine in the file webtest.c

Currently only Basic authentication is supported, MD5 will be in a future release.

A note about implementation: Passing the file descriptor (FD) is very efficient for the embedded FS, but may be less than ideal for the OS disk-based FS. Finding out which file is being passed form the FD requires intimate knowledge of the file system - usually a pain to actually code.

One way around this would be to hook the wi_fopen() member of the wi_filesys structure, and keep a map of file name (and/or other useful access info) indexed by the FD. Another approach would be to modify the code in webio.c to pass more info to the auth routine.

Portability:

For example, on Linux a socket can only be closed by passing it to the file system's close() call. On Windows, a socket is closed by calling Winsock's closesocket() - passing it to close(), at least on some versions of Windows, will result in a system crash. Webio maintains portability across both system by using the less common closesocket() call throughout it's code, and mapping this to close() on Linux ports. Since the syntax of close and closesocket() are identical, this is done with a C #define.. In fact, to support the forgetful and dyslexic, we also have a #define. for socketclose()..

The remainder of this section lists each area where there are deviations between Linux, Windows, and common Embedded targets, and describes how Webio addresses the various systems with one clean code base. The only assumptions are

• A standard C compiler and library.
• A TCP stack with a sockets-like network API.
• Multiple threads, or the ability to periodically poll a core routine.

Portability files and routines

Header files

Portbility routines

WI_NOBLOCKSOCK()

This primitive is passed an open socket, which should be set to non-blocking mode. Once in non-blocking mode, calls to redcv() and send() should return EWOULDBLOCK if they cannot complete immediately. The Linux example:
#define WI_NOBLOCKSOCK(socket) fcntl(socket, F_SETFL, O_NONBLOCK)

wi_getdate()

This primitive should return a static string containing the current date and time in a user-readable format. The returned value will be used in building HTTP headers, so the format must be compliant with HTTP standards. See the Windows example for a usable format string.

strnicmp()

This primitive compares two strings up to a passed length, ignoring case. It is a merging of the ANSI functions stricmp() and strncmp(). Despite it's non-standard status, many C libraries (notably Microsoft) support it anyway. Because of this, Webio ports to systems without it require that it be in the portability code.

closesocket()

This primitive should close the passed open socket. On most Unix-like systems it can be mapped directly to close().

Debugging aids

dtrap()

heap checking

The case for HTML GUIs

This section could be called a rant, but this author has been mystified for 20 years as to why there is no industry standard for implementing GUIs across different Operating Systems. We have an ANSI standard C library, sockets (and quasi-socket Winsock) for Networking. But since the Xerox Star, through Apple's Lisa and MAC, past OS/2, and into Windows and X-Windows, no two popular systems use even similar GUIs - until the advent of HTML.

HTML with CSS and Javascript provides a serviceable GUI across every OS with a mouse and graphics screen. While the displays don't usually support HTML natively, most platforms have at least one free web browser available. Even my cell phone came with one.

With the universal GUI platform, It only makes sense for application writers to implement simple GUIs with HTML, rather than Windows, MAC or Linux GUI calls. While some graphics intensive apps (such paint programs) are still better done on a native GUI, apps such as spreadsheets, text editors, and databases are as easy, if not easier, to implement on HTML. Look at the sophisticated office suite provided by Google apps.

The HTML GUI also has the benefit of freeing the user from having the GUI on the same computer that runs the app. By opening a few ports in a home firewall, you could run all your apps on a machine at your house and use them from any browser, anywhere on the Internet. No VPN or PC-Anywhere-like add-ons required.

The one missing ingredient for most application writers to write such apps is a web server designed to be embedded right in the application as a library - like Webio.