Spelling corrections.
1 2 AOLserver 4.5 - June 2006 3 ------------------------- 4 5 AOLserver 4.5 is a major upgrade including several new Tcl commands, 6 C API's, and resource management improvements. 7 8 9 Contents: 10 -------- 11 12 New and Updated Binaries 13 New Modules 14 New and Updated Tcl Commands 15 New and Updated C Routines 16 Additional Updates 17 Changes to the Build Process 18 Bug Fixes 19 Known Issues 20 21 22 New and Updated Binaries: 23 ------------------------ 24 25 libnsd: 26 The AOLserver library now includes an entry point suitable 27 for loading into an ordinary, thread-enabled, tclsh, e.g.,: 28 29 # tclsh 30 % load /usr/local/aolserver/lib/libnsd.so 31 % ns_time 32 33 This provides the full set of Tcl commands within the 34 AOLserver core although several commands will generate 35 errors for lack of an HTTP request context (e.g., ns_conn). 36 37 nstclsh: 38 A simple Tcl-shell linked against libnsd. It's equivalent 39 to loading libnsd into an ordinary tclsh as described above. 40 41 42 New Modules: 43 ----------- 44 45 nszlib: 46 The popular nszlib module written by Vlad Seryakov has been 47 integrated into the core release with modifications to 48 enable the new Ns_Gzip routine. The module also provides 49 the "ns_zlib" command for compressing and uncompressing 50 strings and files. 51 52 nsproxy: 53 The nsproxy modules provides the "ns_proxy" command which 54 enables sending Tcl scripts to a proxy process connected 55 via a pipe. The proxy includes core Tcl commands as well 56 as AOLserver commands from the libnsd library. Evaluating 57 scripts in a proxy process can be used to isolate and/or 58 timeout thread-unsafe or otherwise unsafe 3rd party code. 59 60 61 New and Updated Tcl Commands: 62 ---------------------------- 63 64 The following Tcl commands are new or updated from the last release. 65 As few of the man pages have yet to be written, see the source code 66 for usage details and options. 67 68 ns_pools: 69 The "ns_pools" command enables configuration of one or more 70 pools of connection processing threads. The pools allow 71 certain requests to be handled by specific threads. This 72 could, for example, ensure multiple long running requests 73 don't block other short running requests. Pools are selected 74 based on method/url pairs similar to the mappings managed 75 by the "ns_register_proc" command. By default, all requests 76 are handled by a single, unlimited, "default" pool. There 77 is also an "error" pool as described below. Coupled with 78 the new "ns_limits" command, pools can provide for sophisticated 79 resource management. 80 81 ns_limit: 82 The "ns_limit" command enables setting various resource 83 limits for specified method/url combinations. These limits 84 include such items as max concurrent connections, max file 85 upload size, and timeouts waiting for connection processing. 86 When limits are exceeded, connections are immediately 87 dispatched to a dedicated "error" connection processing 88 pool to generate a quick error response. By default all 89 requests share the same default limits. Coupled with the 90 new "ns_pools" command, URL-based limits can provide for 91 sophisticated resource management. 92 93 ns_register_fastpath: 94 The new "ns_register_fastpath" command enables mapping of 95 specific directories to the internal static file response 96 code. See also the new Ns_FastPathOp API. 97 98 ns_adp_flush: 99 The "ns_adp_flush" command allows buffered ADP output to 100 be immediately flushed to the client. If no output has been 101 sent, the server will first generate the appropriate HTTP 102 response headers for a streaming response. An implicit 103 flush will also be performed when the ADP buffer overflows. 104 105 ns_adp_close: 106 The "ns_adp_close" command flushes content as with ns_adp_flush 107 and then closes the connection so no additional content can 108 be sent. This can be useful to keep the client from waiting 109 while post-processing steps continue. 110 111 ns_adp_registerscript 112 The "ns_adp_registerscript" command is similar to the 113 ns_adp_regsiteradp command except it registers an ordinary 114 script instead of text/script ADP code to execute for a 115 given tag. 116 117 ns_adp_ictl: 118 The new "ns_adp_ictl" command provides various options to 119 manage the ADP execution environment for current Tcl 120 interpreter, e.g.: 121 122 ns_adp_ctl bufsize ?size?; # Max output before forced flush 123 ns_adp_ctl autoabort ?bool?; # Abort execution on client drop? 124 ns_adp_ctl singlescript ?bool?; # Compile pages as a single script? 125 ns_adp_ctl gzip ?bool?; # Gzip output if enabled in server? 126 ns_adp_ctl trimspace ?bool?; # Trim leading whitespace in response? 127 ns_adp_ctl channel ?chan?; # Set channel for ADP output 128 ns_adp_ctl trace ?bool?; # Trace ADP execution 129 ns_adp_ctl detailerror ?bool?; # Generate detailed error messages 130 ns_adp_ctl displayerror ?bool?; # Format error message in page output 131 ns_adp_ctl stricterror ?bool?; # Abort ADP execution on flush error 132 133 ns_adp_include: 134 The "ns_adp_include" command has been enhanced with the new 135 "-cache ttl" and "-nocache" options to support execution 136 results caching. Applications which isolate fully dynamic 137 page components (e.g., personalized content) from partially 138 dynamic page components (e.g., slowly changing news headlines) 139 can benefit from using these options to cache and reuse the 140 results of slowly changing components. This approach has 141 the potential to dramatically improve the performance of 142 such applications. 143 144 ns_adp_ident: 145 The new "ns_adp_ident" command can be used to tag and query 146 ADP file versions. 147 148 ns_cache: 149 The "ns_cache" command enables creation and management of 150 memory-based caches utilizing the underlying Ns_Cache C 151 API's. 152 153 ns_driver 154 The new "ns_driver" command provides a "list" option to 155 enumerate all driver instances and a "query" option to 156 return a Tcl list of various statistics for a given driver. 157 158 ns_ictl: 159 The "ns_ictl" command has been enhanced with several new 160 options including: 161 162 trace: 163 Enables Tcl-script based callbacks at points defined 164 by the Ns_TclRegisterTrace routine. 165 166 threads: 167 Returns a list of all threads with Tcl interps for 168 current virtual server. 169 170 cancel: 171 Sends an async-cancel message to a given thread, 172 interrupting any Tcl execution in any AOLserver-based 173 interpreter executing on the thread. 174 175 package: 176 Registers a Tcl package to be automatically required 177 in each interp for the current virtual server. 178 179 once: 180 Registers a script to be executed exactly once for 181 a virtual server. 182 183 ns_internalredirect: 184 The "ns_internalredirect" command enables re-authorization 185 and restart of a connection as if it where originally for 186 a different URL. This command provides access to the 187 previously private routines used internally for such purposes, 188 e.g., by the fastpath to handle directory URL's mapped to 189 an ADP index files such as "index.adp". 190 191 ns_for: 192 ns_foreach: 193 ns_while: 194 These commands are replacements for the standard "for", 195 "foreach", and "while" command which have the same semantics 196 but also allow for monitoring and control by the "ns_loop_ctl" 197 command described below. 198 199 ns_loop_ctl: 200 The "ns_loop_ctl" command enables listing, pause, resume, 201 cancel, and evaluation of a script within "ns_for", 202 "ns_foreach", and "ns_while" loops executing in other 203 threads. In addition, the "install" option can be used at 204 to swap the "ns_for", "ns_foreach", and "ns_while" commands for 205 the "for", "foreach", and "while" commands, enabling monitoring 206 of all loops in unmodified code. 207 208 ns_register_encoding 209 The new "ns_register_encoding" command enables mapping of 210 method/URL combinations to specific charset encodings used 211 to decode the request. If no mapping exists for a given 212 method/URL, the server will use that defined for the virtual 213 server or process wide by the "urlcharset" config setting. 214 This setting defaults to NULL, or no encoding, thus assuming 215 UTF-8 characters in the request data. 216 217 ns_returnmoved: 218 The "ns_returnmoved" command is similar to the "ns_returnredirect" 219 command but returns a 301 "permanently moved" HTTP status 220 code along with the redirected location. 221 222 223 New and Updated C Routines: 224 -------------------------- 225 226 The following routines are new or updated from the last release. 227 As few of the man pages have yet to be written, see the source code 228 for parameter and results details. 229 230 Ns_BinPath: 231 New routine to construct file pathnames relative to the 232 bin/ subdirectory similar to the Ns_LibPath and Ns_HomePath 233 routines. 234 235 Ns_ConnContent: 236 Return a pointer to request content. If the content is not 237 in memory and has been spooled to a temp file, a new file 238 mapping will be created and returned. Note that the content 239 is likely, but not guaranteed, to be followed by a terminating 240 null byte. See also the "content" option of the "ns_conn" 241 command. 242 243 Ns_ConnContentFd: 244 Return a file descriptor open on a temp file containing 245 request content. If the content is in memory only, it will 246 first be copied to a newly allocated temp file. You must 247 not close the returned fd as it is owned by the server core 248 which will reuse the open file for subsequent requests. 249 For an example, see nscgi/nscgi.c which has been updated 250 to use Ns_ConnContentFd instead of spooling to a temp file 251 directly. See also the "contentchannel" option to the 252 "ns_conn" command which returns a duplicate of the fd 253 returned via Ns_ConnContentFd registered as a Tcl channel. 254 255 Ns_ConnContentOnDisk: 256 Returns 1 if the request content is already located in a 257 temp file or 0 if the content is located in memory. A call 258 to Ns_ConnContent after Ns_ConnContentOnDisk has returned 259 1 will result in a new file memory mapping being created. 260 261 Ns_ConnGetFile: 262 Ns_ConnFirstFile: 263 Ns_ConnNextFile: 264 Routines to find a specific file or enumerate all file 265 objects embedded within a multipart form. See also the 266 "files" option to the "ns_conn" command. 267 268 Ns_ConnSendFdEx: 269 Send contents of an open file descriptor from a specific 270 file offset. This routine uses pread on Unix to enable 271 multiple simultaneous calls with the same open fd and is 272 currently unsupported on Win32. 273 274 Ns_ConnReturnOpenFdEx: 275 Generate a complete response with content from open file 276 descriptor at given offset. This routine uses Ns_ConnSendFdEx 277 and as such is also unsupported on Win32. 278 279 Ns_ConnReturnServiceUnavailable: 280 New routine to return an HTTP 503 response. 281 282 Ns_CheckStack: 283 Checks if the stack appears to have overrun. Note that the 284 ADP evaluation engine now calls Ns_CheckStack on entry to 285 each call frame and will generate an exception in the case 286 of an endlessly recursive ADP condition. 287 288 Ns_ClsAlloc: 289 Ns_ClsSet: 290 Ns_ClsGet: 291 New interface, similar to thread-local storage, which enables 292 user data to be saved and retrieved within the connection 293 structure. This interface is intended to be used with the 294 Ns_QueueWait interface so that data can be saved in the 295 driver thread and retrieved later in a connection processing 296 thread. 297 298 Ns_ConnSetStatus: 299 Ns_ConnGetStatus: 300 New routines to set and get the HTTP status code to be used 301 in responses generated via Ns_ConnFlush. See also the 302 "status" option to the "ns_conn" command. 303 304 Ns_ConnSetType: 305 Enhanced to more intelligently set the input and output 306 character encodings based on intended output charset. See 307 also the "ns_adp_mimetype" command and the "encoding" and 308 "urlencoding" options to the "ns_conn" command. 309 310 Ns_ConnGetGzipFlag: 311 Ns_ConnSetGzipFlag: 312 Ns_ConnGetKeepAliveFlag: 313 Ns_ConnSetKeepAliveFlag: 314 New utility routines to query or set the gzip and connection 315 keep-alive advisory flags. 316 317 Ns_ConnFlush: 318 New general purpose routine to construct and return dynamically 319 generated, text-based responses, e.g., as generated from 320 ADP. The routine incorporates much of the logic previously 321 embedded in the ADP interface including the ability to 322 automatically covert from UTF-8 to the output encoding 323 and/or gzip-compress the output. In addition, the routine 324 supports streaming responses, generating headers on the 325 first call followed by HTTP/1.1-style chunked-encoding 326 blocks on subsequent calls. See also the "ns_adp_flush" and 327 "ns_adp_close" commands. 328 329 Ns_ConnFlushDirect: 330 Similar to Ns_ConnFlush but without any modification of the 331 given buffer, thus useful for non-text content. 332 333 Ns_DevNull: 334 Returns a shared file descriptor open on /dev/null. This 335 fd is owned by the AOLserver process and must not be closed. 336 337 Ns_FastPathOp: 338 New request callback to utilize the internal "fastpath" 339 caching-file response code. See also the new "ns_register_fastpath" 340 command. 341 342 Ns_AdpRequestEx: 343 Extended version of Ns_AdpRequest with an additional Ns_Time 344 argument, specifying the timeout of optional execution 345 results caching. 346 347 Ns_FindCharset 348 Locates the charset string within a full mimetype string, 349 e.g., locating "utf-8" in "text/html; charset=utf-8". 350 351 Ns_Gzip: 352 Ns_SetGzipProc: 353 Routine to gzip-compress a buffer if a compression function 354 has been registered with a call to Ns_SetGzipProc. The new 355 "nszlib" module registers such a callback. 356 357 Ns_GetAllAddrByHost: 358 New routine to return all addresses for a given host. See 359 also the "ns_addrbyhost -all" command. 360 361 Ns_ParseRequestEx: 362 New extended version of Ns_ParseRequest which takes an 363 optional Tcl_Encoding argument specifying the charset to 364 decode the URL. The default is based on the context of the 365 request. 366 367 Ns_TaskCreate: 368 Ns_TaskEnqueue: 369 Ns_TaskRun: 370 Ns_TaskCancel: 371 Ns_TaskWait: 372 Ns_TaskCallback: 373 Ns_TaskDone: 374 Ns_TaskFree: 375 Ns_CreateTaskQueue: 376 Ns_DestroyTaskQueue: 377 New facility for event-based callback queues designed to 378 replace the legacy Ns_SockCallback interface. As an example, 379 see the "ns_http" command which has been updated to use the 380 task interface. 381 382 Ns_TclRegisterTrace: 383 New routine to registere callbacks at specific points in 384 the lifetime of a Tcl interpreter. This routine is a 385 replacement for the legacy Ns_TclRegisterAt* routines. See 386 also the "trace", "runtraces", and "gettraces", options to 387 the "ns_ictl" command. 388 389 Ns_QueueWait: 390 New interface to enable event-driven callbacks in the driver 391 thread before dispatching to pools for processing. This 392 allows drivers to augment data received from the client 393 (headers, request, content) with additional data fetched 394 over the network (likely stored in connection-local storage 395 via Ns_Cls). An example would be to add certain personalization 396 data received via a Web service. The benefit of this approach 397 is to accumulate additional data efficiently with driver-thread 398 based event callbacks instead of through potentially blocking 399 calls within connection threads. 400 401 Ns_SockWaitEx: 402 New extended version of Ns_SockWait using higher-resolution 403 millisecond timeouts instead of 1-second resolution. 404 405 ns_poll: 406 New wrapper around system-provided poll or an emulation 407 based on select when poll isn't available (Win32) or broken 408 (OS/X). 409 410 Ns_LibInit: 411 NsThreads_LibInit: 412 New routines to initialize the libnsd and libnsthread 413 libraries. It is normally not necessary to call these 414 routines as the dynamic library entry points as well as the 415 Ns_Main and Ns_TclInit routines call these init routines 416 at startup. They are available if necessary to initialize 417 specialized, statically linked applications. 418 419 420 Additional Updates: 421 ------------------ 422 423 * Enabled a global, pseudo-server, to support all Tcl commands 424 outside the context of a virtual server, e.g., when the libnsd.so 425 library is loaded into a Tcl shell. 426 427 * Updated Tcl trace execution order to be FIFO and LIFO for init 428 and cleanup callbacks respectively. 429 430 * Moved more of the legacy (and generally confusing) Tcl interp 431 duplication code from C into the the bootstrap init.tcl script 432 utilizing the ns_ictl command. 433 434 * ADP output buffer is now configurable with a maximum size (see 435 the "ns_adp_ctl" command above). Upon overflow, the current output 436 will be flushed to the client immediately in streaming mode to avoid 437 consuming unbounded memory. 438 439 * Restored AOLserver 3.x ADP parsing order, based on a patch from 440 Alastair Young, where registered tags, <% ... %> scripts, and 441 <script> ... </script> tags all have the same precedence. The ADP 442 parser now also properly handles nested sequences, e.g.: 443 444 <% ns_adp_puts [ns_adp_eval {<% ... %>} %> 445 446 * Enhanced ADP error messaging capabilities which can be managed 447 on a per-interp basis with ns_adp_ctl or configured by default, 448 e.g.: 449 450 ns_section "ns/server/server1/adp" 451 ns_param stricterror false ;# Stop execution on error 452 ns_param detailerror true ;# Include connection detail 453 ns_param displayerror false ;# Return error message to client 454 455 When "detailerror" and/or "displayerror" are set, an error message 456 similar to the following would be appended to the server log and/or 457 included in the output sent to the client: 458 459 This is an error. 460 while executing 461 "error "This is an error."" 462 at line 4 of adp file "/aolserver/servers/server1/pages/error.adp" 463 while processing connection #2: 464 GET /error.adp HTTP/1.1 465 Accept: */* 466 Accept-Language: en 467 Accept-Encoding: gzip, deflate 468 User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) ... 469 Connection: keep-alive 470 Host: 192.168.2.1:8000 471 472 * New ADP execution tracing which can be enabled for a request with 473 the ns_adp_ctl command or configured by default, e.g.: 474 475 ns_section "ns/server/server1/adp" 476 ns_param trace false ;# Enable text/script debug info. 477 ns_param tracesize 40 ;# Max chars of each chunk to log. 478 479 The output of ADP tracing is somewhat limited and poorly formatted. 480 481 * Communication driver threads are now dedicated to each instance 482 of loaded drivers instead of a single thread multiplexing all I/O. 483 This both simplified the code and enabled the new Ns_QueueWait API. 484 485 * Communication driver threads now truncate instead of freeing I/O 486 dstring buffers, avoiding repeated realloc calls between connections. 487 * Re-ordered inits to ensure startup fds and log file are initialized 488 first. 489 490 * Communication drivers and connection data access routines have 491 been updated to support large request content spooled to temp files 492 instead of always copying to potentially large in-memory regions. 493 See the Ns_ConnContentFd API for more details. 494 495 * Updated to HTTP response version (1.0 or 1.1) to match client 496 version. 497 498 * Updated "ns_db" Tcl API to be a native Tcl_Obj command which 499 should improve performance over the previous technique of multiple 500 comparisons of command option strings. 501 502 * Added debug driver configuration option which will enable logging 503 of unusual I/O or protocol errors. 504 505 * Simplified charset/encoding config for process-wide url decoding 506 and per-virtual server defaults for dynamic Tcl requests. 507 508 * Several new default file extension to mimetype mappings have been 509 added in nsd/mimetype.c, e.g., .mp3 = audio/mpeg. 510 511 * Created new default index.adp page which displays a number of 512 AOLserver configuration settings. 513 514 * The OS/X dynamic library loading code has been removed in favor 515 of the more standard dlopen/dlsym API now provided by OS/X 10.4. 516 517 518 Changes to the Build Process: 519 ---------------------------- 520 521 * The Unix and Win32 build environments have been tightly integrated. 522 Basically, the makefiles support both gmake on Unix and nmake on 523 Win32 by including a platform-specific include/ns.mak created by 524 the nsconfig.tcl Tcl script. As a side effect, several makefile 525 variable names had to be changed, e.g., "LIB" on Unix is now "DLL" 526 on Unix and Win32. In addition, platform-independent Tcl scripts 527 are used for file removal and copying, a further dependency on a 528 working Tcl installation while building. 529 530 * On Unix, backwards compatible Makefile.module and Makefile.global 531 are provided which simply include the new ns.mak and map old variables 532 to new names if present, e.g., "LIB" to "DLL". 533 534 * On Win32, the legacy Visual Studio build directories have been 535 removed entirely has their maintenance proved irritating and 536 complicated. 537 538 * A new util/nsinstall-man.sh script installs man pages and also 539 optionally converts the pages to HTML using groff on Unix. 540 541 * Several new makefile install directives have been added, including 542 install-tcl, install-skel, install-util, install-bins, and install-docs. 543 544 * Added new stubs and makefile targets to support building a library 545 or module in a consistent way which separates the library code 546 installed as both a static archive and dynamic lib from the library 547 and/or module init calls. 548 549 * Added mynsd example directory showing how to build a custom static 550 linked nsd binary. The previous conditional startup code and 551 makefile support has been removed. 552 553 * Compiling for 64-bit is known to work in general on Unix. It is 554 unclear if AOLserver has ever been successfully compiled on 64-bit 555 Windows and there is a known issue with the Win32 thread interface 556 which would appear to make this not possible (the thread exit status 557 pointer is cast to an smaller-sized integer). In addition, while 558 AOLserver runs on 64-bit Unix systems, the interfaces have not been 559 scrubbed and updated to support all 64-bit features, e.g., large 560 file offsets. A more extensive 64-bit certification effort is planned 561 for the next release. 562 563 * Applied patches from Stephen Deasey for better compile time error 564 checking, e.g., now checking printf-style arguments in Ns_DStringPrintf. 565 566 567 Bug Fixes: 568 --------- 569 570 * Fixed crash bug caused by certain command sequences generated 571 when using the "ns_chan" Tcl API (SF #1094480). 572 573 * Fixed crash bug in channel tables cleanup and closing code. 574 575 * Fixed crash bug loading nsdb outside a virtual server. 576 577 * Fixed invalid return code in Ns_ConnReadLine(). 578 579 * Updated simple HTML response routines (e.g., Ns_ConnReturnNotice, 580 ns_returnnotice) to properly encode output from UTF-8. 581 582 * Fixed bug where log level "dev" wasn't properly supported. 583 584 * Switched to strtol() from atoi() for improved thread safety. 585 586 * Added check for Darwin (OS/X) to disable detection of poll() which 587 appears broken, using the poll() emulation code based on select() 588 instead. 589 590 * Fixed infinite recursion crash bug in GetAddr(). 591 592 * Removed minor memory leak when no CGI mappings are present. 593 594 * Removed most panics when compiling on 64 bit machines. 595 596 * A thread race condition which could lead to a crash when using 597 "ns_server threads" has been fixed. 598 599 600 Known Issues: 601 ------------ 602 603 * Documentation remains incomplete. We are still missing useful 604 overview and getting started materials and, while significant 605 progress has been made, most of the man pages are still empty stubs. 606 607 * Loading libnsd into a tclsh and then creating new threads with 608 the ns_thread command will result in a crash when those threads 609 exit. The issues has to do with finalization of the async-cancel 610 context used to support the new "ns_ictl cancel" feature. This bug 611 is not present when using the "nstclsh" binary. 612 613 * The Win32 port does not support all of the new features of temp-file 614 based request content. In particular, while large requests will 615 spool to a temp file, a subsequent call to Ns_ConnContent will fail 616 as the corresponding file mapping code has not yet been written. 617 618 * The build environment in general is annoying. As described, there 619 was an effort in this release to support Unix and Win32 from the 620 same makefiles which, while a bit clever, likely irritated more 621 Unix developers than excited Win32 developers. In addition, the 622 Unix configure script is weak, overly reliant on the context from 623 a properly built Tcl installation, and the makefiles rely too much 624 on GNU make features. Creating a proper Unix configure script which 625 creates proper Makefiles is planned for the next release along with 626 a possible shift in the Win32 strategy (see next note). 627 628 * AOLserver does not compile with the latest Visual C++ 2005 from 629 Microsoft. While some of the new safety and security features in 630 Visual C++ are interesting and the debugger has always been excellent, 631 the complexity of Visual Studio and incompatibility of the Microsoft 632 command line tools with Unix equivalents makes supporting Win32 633 increasingly irritating. As such, we will be exploring a shift to 634 Msys-based Win32 build in the next release, enabling us to utilize 635 our Unix-based development approach while still allowing AOLserver 636 to run against native Win32 libraries. Feedback is appreciated. 637 For now, AOLserver can be compiled on Win32 as described in the 638 README file using the previous release of Visual C++.
Copyright © 2010 Geeknet, Inc. All rights reserved. Terms of Use