Modal Web Server Example Part 4

This document follows on from Part One, Part Two and Part Three. You should read those documents first to get an understanding of this one. The source code for this part is in modal-web-server-0.4.tar.gz.

Overview

First we discuss a bug that has surfaced and work through how to fix it. Then we extend the current framework to allow attaching 'callback' functions that get called when form elements are processed.

Bug Fix

One fairly major bug has appeared in previous versions of the framework and fixing it gives me a chance to write about unintentionally capturing state in a continuation.

The bug appears when resuming a continuation from a different browser or machine than the one that originally caused it to be created.

It can be demonstrated by going to the URL for the show-message example from part one. Following the steps below:

  1. Start the show-message example
  2. Press 'Ok'
  3. Copy the URL and paste it into another browser window

Once pasted in the second browser window a request is made to the server for that continuation again. The continuation is returned from the registry and entered. At this point the browser may appear to 'hang' or get no answer from the request. Or it may work. The result is sometimes random.

The problem is caused by the HTTP library in the server binding the 'current-output-port' and 'current-input-port' to the output and input ports for the socket used for the connection to the browser. This binding is done on the very first request which then captures the continuation for when the 'Ok' URL is requested. The Chicken Scheme HTTP code that does this is:

(define (handle-request req in out)
  (let ([m (http:request-method req)])
    (dribble "handling ~A request..." m)
    (let ([h (http:request-method-handler m)])
      (if h
	  (parameterize ([current-input-port in]
			 [current-output-port out] )
	    (h req) )
	  (error "undefined request method handler" m (http:request-url req))))))

For normal operation of the web server this is fine, but because we are capturing continutions and resuming them upon future requests it causes problems.

When 'Ok' is pressed in our example, the continuation is resumed but the binding of the input and output ports is still to the old socket connection from the original request. This is unlikely to still be open and causes an error. Which in our limited error checking framework causes the thread to die, resulting in the browser not getting an answer to its request.

The fix is easy. We need to not use the current-input-port and current-output-port from the re-entered continuation. Instead when the request handler is entered we save the values of these variables to a parameter (ie. a thread local variable). We always use the values stored in the parameters for sending data across the socket.

With this change, when requesting the 'Ok' URL, the request handler first sets the parameters for the ports to be that of the current-output-port and current-input-port (which are those for the current socket connection) and then calls the continuation. The continuation will use these values (the current socket connection) rather than the old values stored in current-output-port and current-input-port.

Here's some example code that does this. The important bit is in red:

(define (show page)
  (redirect-to-here)
  (process-callbacks
   (call/cc
    (lambda (k)
      (let ((kid (get-unique-continuation-id)))
        (hash-table-set! kid-registry kid k)
        (http:add-resource
         (string-append "/" (symbol->string kid))
         (lambda (r a)
           (response-input-port (current-input-port))
           (response-output-port (current-output-port))
           (call/cc
            (lambda (exit)
              (suicide exit)
              (let ((k (hash-table-ref kid-registry kid)))
                (k (http:request-body r)))))))
        (process-page-function-result (page (symbol->string kid)))
        ((suicide) #f))))))

As mentioned before, the 'response-input-port' and 'response-output-port' are parameters. They are set to the current socket values immediately upon entering the request handler. They are initially set to default to 'current-input-port' and 'current-output-port' to enable calling the functions from the read-eval-print loop to output to the console for debugging reasons:

(define response-input-port (make-parameter (current-input-port)))
(define response-output-port (make-parameter (current-output-port)))

The code that outputs data to the socket is changed to use these parameters instead of the defaults:

(define (process-page-function-result result)
  (when result
    (http:write-response-header 200 "OK" '() (response-output-port))
    (display
     (format "Content-type: text/html\r\nContent-length: ~a\r\n\r\n~a" 
             (string-length result) 
             result)
     (response-output-port))))

With these changes the bug is fixed. Whenever a continuation is captured you need to be aware of what state may be captured at the time and whether that state is 're-usable'. In Scheme systems that support serializing continuations you also need to be aware if the captured state is serializable. If it is not then the continuation cannot be serialized.

Callbacks

The examples presented so far have mainly followed the traditional way of processing HTML forms. The data is presented as name/value pairs which are processed once the form is submitted by the browser.

We're now going to add a simple 'callback' system. It allows us to associate functions that get called when form items are submitted. This works in a similar manner to 'function-href' which we used in part one.

'function-href' enabled us to associate a function to an URL. That function got called when the URL was requested by the user by clicking it in the browser. We'll extend the framework to add the ability to associate functions that will be called when individual form items are submitted. The value of those form items will be passed as an argument to the function. This makes it easy to set application structures (instances of classes, etc) to values contained within a form.

In this part I'll start off with a relatively simple implementation, discuss the strengths and weaknesses, and move to a more robust solution in a later part. I'll start off showing a couple of examples of usage and then describe how it is implemented.

Example One

The first example requests the user to enter three numbers, each number on a seperate page. These numbers are appended to a list held as a local variable. When the list is greater or equal to three numbers the final page displays the three numbers. It's a contrived example but demonstrates modifying local state via the callback.

The input form to ask for a number holds a single INPUT item, of type 'text'. Instead of giving it a name we use a new function in the framework to generate a unique name for the INPUT item. A callback function is associated with this unique name. When the form is submitted, all the names of the form items are checked to see if a callback is associated with it. If it is, the callback is called, passing it the value of that form item.

Here's the 'show' call for requesting a number:

(show
 (lambda (url)
   (sxml->html-string
    `(html
      (head (title "Enter a number"))
      (body
       (p "Enter a number: ")
       (form (@ (action ,url) (method "post"))
             ,(text-input ""
                          (lambda (v)
                            (set! numbers
                                  (cons (string->number v) numbers))))
             (input (@ (type "submit")))))))))

The portion in red shows the setting of the callback and the creation of the text input element. Here we create a text element with a random unique name. When the form is submitted the values of that text element (The number entered by the user) is passed to the callback. This callback sets the 'numbers' local variable to have the number entered appended to the beginning of the list.

The 'text-input' function is:

;; Return an HTML fragment for a form text input field. When the form
;; for this field is submitted, call the callback function with the
;; value entered in the field. 
(define (text-input value callback)
  (let ((id (get-unique-field-id)))
    (register-form-callback! id callback)
    `(input (@ (type "text") 
               (value ,value) 
               (name ,(number->string id))))))

This returns a simple HTML INPUT element with the requested values. Now there is no need to check the return value of the 'show' call, extract the fields out of it, and store the values in the list. It can all be done inside the function to produce the HTML page itself making everything self contained.

The entire function to request the three numbers and display the result is:

(define (get-three-numbers)
  (let ((numbers '()))
    (let loop ()
      (if (>= (length numbers) 3)
          (show-message (format "The three numbers were: ~a ~a ~a"
                                (first numbers)
                                (second numbers)
                                (third numbers)))
          (begin
            (show
             (lambda (url)
               (sxml->html-string
                `(html
                  (head (title "Enter a number"))
                  (body
                   (p "Enter a number: ")
                   (form (@ (action ,url) (method "post"))
                         ,(text-input ""
                                      (lambda (v) 
                                        (set! numbers (cons (string->number v) numbers))))
                         (input (@ (type "submit")))))))))
            (loop))))))

And it can be registered and run with:

(register-function get-three-numbers)

Example Two

This example provides an HTML interface to a simple 'database' of people.

Data Model

We use the Scheme SRFI-9 record types to create a 'person' type:

;; A 'person' type used for holding data related to a particular person.
;; This data will be collected via an HTML form.
(define-record-type :person
  (person id name address1 address2 address3 phone)
  person?
  (id get-id set-id!)
  (name get-name set-name!)
  (address1 get-address1 set-address1!)
  (address2 get-address2 set-address2!)
  (address3 get-address3 set-address3!)
  (phone get-phone set-phone!))

A person can be created using 'make-new-person'. It is assigned a unique 'id' and stored in a list of people:

;; Create a person with default values
(define (make-new-person)
  (person #f "" "" "" "" ""))

A database of people (implemented as a list) is kept, along with a function to add people to the database. A mutex (from SRFI-18) is used to prevent multi-user access from corrupting the database

;; We are keeping a database of 'people' that will be accessed from
;; multiple threads. Use a mutex to prevent problems with accessing
;; the global 'database'.
(define person-mutex (make-mutex))
(define person-database '())

;; Store the person in the database
(define store-person
  (let ((count 0))
    (lambda (person)
      (mutex-lock! person-mutex)
      (set-id! person count)
      (set! person-database (cons person person-database))
      (++! count)
      (mutex-unlock! person-mutex))))

To find a person with a particular id we search through the list:

(define (find-person id)
  (mutex-lock! person-mutex)
  (let ((result
         (call/cc
          (lambda (return)
            (for-each
             (lambda (person)
               (when (= (get-id person) id)
                 (return person)))
             person-database)
            #f))))
    (mutex-unlock! person-mutex)
    result))

HTML View

Our HTML interface will display a list of people in the database and allow the user to select and person and edit their details, add a new person or delete an existing person.

To display the list of people in the database we can use a number of HTML approaches.

To start with we will list each persons details as a row in a table. The 'id' for the person will be linked to a function that allows the user to edit the persons details. A single anchor at the end of the table will allow adding additional people.

;; An HTML view of all people in the database presented as a table.
(define (show-people-in-table)
  (let loop ()
    (show
     (lambda (url)
       (sxml->html-string
        `(html
          (head (title "Database of People"))
          (body
           (table (@ (border "1"))
                  (tr (th "ID")
                      (th "Name")
                      (th "Address1")
                      (th "Address2")
                      (th "Address3")
                      (th "Phone"))
                  ,@(map
                     (lambda (person)
                       `(tr
                         (td ,(function-href
                               (number->string (get-id person))
                               (lambda ()
                                 (edit-person person)
                                 (loop))))
                         (td ,(get-name person))
                         (td ,(get-address1 person))
                         (td ,(get-address2 person))
                         (td ,(get-address3 person))
                         (td ,(get-phone person))))
                     person-database))
           (p ,(function-href "Add New Person"
                              (lambda ()
                                (add-new-person)
                                (loop)))))))))
    (loop)))

Most of this function should be pretty familiar by now. I've used 'function-href' to provide the link from the 'id' to the function which displays the editing user interface, and on the 'Add New Person' anchor.

One downside with this technique is it generates a lot of continuations. If there are 100 people, then 100 continuations will be generated for the 'edit' link. If a 'remove' link was provided this would be another 100. Later on I present an approach which works around this.

To implement the 'add-new-person' I call 'make-new-person' to create a new person and then call 'edit-person' to allow editing the details. After the 'edit-person' form is submitted by the user the person is stored in the database with 'store-person'. It is done in this order to allow the user to use the browser 'back' button on the edit page to go back to the main list of people without adding the person to the database. This effectively cancels the request to add the new person.

;; Create a new person and display an HTML page allowing their details to be edited
(define (add-new-person)
  (let ((person (make-new-person)))
    (edit-person person)
    (store-person person)))                 

If I had called 'store-person' before 'edit-person' then the person would be added to the database before the user hits 'back'. This would result in an 'empty' person appearing in the list when the user refreshes the list. This is because our local data does not get 'backtracked' when the user hits back.

Finally we get to the 'edit-person' function which demonstrates the callback usage again.

;; Display an HTML page allowing a persons details to be edited
(define (edit-person person)
  (show
   (lambda (url)
     (sxml->html-string
      `(html
        (head (title "Person Details"))
        (body
         (form (@ (method "post") (action ,url))
               (table (@ (border "1"))
                      (tr (td "Name")
                          (td ,(text-input (get-name person) (lambda (v) (set-name! person v)))))
                      (tr (td "Address1")
                          (td ,(text-input (get-address1 person) (lambda (v) (set-address1! person v)))))
                      (tr (td "Address2")
                          (td ,(text-input (get-address2 person) (lambda (v) (set-address2! person v)))))
                      (tr (td "Address3")
                          (td ,(text-input (get-address3 person) (lambda (v) (set-address3! person v)))))
                      (tr (td "Phone")
                          (td ,(text-input (get-phone person) (lambda (v) (set-phone! person v))))))
               (input (@ (type "submit") (value "Ok"))))))))))

I'm using the same 'text-input' function defined previously to display the input box for entering details. The default value for the input box is the current value of the field and the callback sets the value in the person record. There is now no need to capture the return value of 'show' and extract each field out individually.

Registering the person database application is done with 'register-function' as usual:

(register-function show-people-in-table)

More effective use of callbacks can be made and I'll demonstrate this in future examples as I extend the framework to add callbacks to other form elements. I'll also provide another way of implementing the 'show-people-in-table' screen and add error handling.

Implementation

Each form element that has a callback associated with it has a unique name. In this implementation the name is a simple incremented integer value:

;; Return a unique id for each field displayed in a form that has a
;; registered callback.  In a 'production' framework
;; this should be unguessable.
(define form-callback-count 0)
(define (get-unique-field-id)
  (let ((id form-callback-count))
    (++! form-callback-count)
    id))

The mapping between the name and the function callback is held in a hashtable:

;; Hashtable containing a mapping between a unique form field id and
;; the callback function for that form field.
(define form-callbacks (make-hash-table))

;; Register a callback for the form field with the given unique id.
(define (register-form-callback! id callback)
  (hash-table-set! form-callbacks id callback))

;; Return the callback registered for the field with the given id
(define (get-form-callback id)
  (hash-table-ref form-callbacks id))

The 'show' function used to return the form results in as a list of pairs. It now processes the results using 'process-callbacks' first, before returning the list.

(define (show page)
  (redirect-to-here)
  (process-callbacks
   (call/cc
    (lambda (k)
      (let ((kid (get-unique-continuation-id)))
        (hash-table-set! kid-registry kid k)
        (http:add-resource
         (string-append "/" (symbol->string kid))
         (lambda (r a)
           (response-input-port (current-input-port))
           (response-output-port (current-output-port))
           (call/cc
            (lambda (exit)
              (suicide exit)
              (let ((k (hash-table-ref kid-registry kid)))
                (k (http:request-body r)))))))
        (process-page-function-result (page (symbol->string kid)))
        ((suicide) #f))))))

'process-callbacks' works through each submitted form item returned from 'show', looks up the global hashtable to see if a callback exists for that form and calls the callback with the value of the form field if it exists.

;; Given the results returned from a continuation resumption (ie. form
;; results), process the callback data. 
(define (process-callbacks form-results)
  (when form-results
    (for-each
     (lambda (field)
       (let ((callback (get-form-callback (string->number (car field)))))
         (when callback
           (callback (cdr field)))))
     form-results))
  form-results)

One problem with this approach that would need to be fixed in a production environment is the fact that the global registry of callbacks will grow forever. There is no current way to expire the information in it. This is a general problem with the example framework presented here and I will probably address it in a later example.