0.13.0

Kristaps Dzonsons

2021-09-23

0.12.5

Kristaps Dzonsons

2021-05-08

0.12.4

Kristaps Dzonsons

2021-03-02

0.12.3

Kristaps Dzonsons

2021-01-20

0.12.2

Kristaps Dzonsons

2020-07-24

Bug fix: some errors encountered writing connection file-descriptors to child processors in FastCGI were accidentally ignored. This came because the function returned success, failure, or hangup, while the caller was testing for failure/success. No such error has ever been reported in the wild—it would probably signify much larger issues that would soon be caught anyway—but the fix is still valid.

Significant simplification of the back-end in removing several crufty interfaces. Back-end logging now uses the front-end logging functions for consistency and to avoid having separate loggers. The same applies to the debugging parameter passed into khttp_parse(3).

Bug fix for FreeBSD with Capsicum when logging to a file opened with kutil_openlog(3). Same goes with the Linux and Darwin sandboxes.

0.12.1

Kristaps Dzonsons

2020-05-11

Deprecate kutil_date2epoch, kutil_date_check, kutil_datetime2epoch, and kutil_datetime_check in favour of khttp_datetime2epoch(3) and khttp_date2epoch(3). These variants use portable implementations of gmtime(3) and timegm(3) that are not encumbered by per-system constraints such as FreeBSD not accepting years prior to 1900 and of course the 32-bit problem.

Add khttp_epoch2datetime(3) to symmetrise khttp_datetime2epoch(3).

Further deprecate kutil_epoch2str, kutil_epoch2utcstr, kutil_epoch2tmvals, and KUTIL_EPOCH2TM as khttp_epoch2str, khttp_epoch2ustr, khttp_epoch2tms, and KHTTP_EPOCH2TM, respectively. The new forms, besides having consistent naming, specifically account for corner cases like negative dates, years with more than four digits, etc. These no longer use the system strtime(3) due to inconsistencies between implementations (e.g., Oracle Solaris not printing >4 digit years) and the 32-bit problem.

0.12.0

Kristaps Dzonsons

2020-04-10

Add the khttp_urlpart(3) family to replace the now-deprecated kutil_urlpart functions. These are for the most part a re-name of the old functions that remove the unused struct kreq argument. There are some small behaviour changes from the original in corner case usage: they have an empty suffix (not just NULL) inhibit printing the dot-suffix, allow a NULL page, and have an empty or NULL page also inhibit the suffix. This way, these functions only produce valid URLs, and also allow for some previously-disallowed (but valid) forms such as /?foo=bar.

The previous functions have been retained with the original behaviour with one exception: before, an empty suffix would still print the period separator. Now, this is suppressed.

The khttp_vurlpart and khttp_vurlpartx forms, which accept a variable-length type, are also now exposed for use.

Add the khttp_urlabs(3) function, which is similar to the earlier kutil_urlabs but significantly more robust and accepts query string arguments. The earlier kutil_urlabs is retained, but deprecated.

Add khttp_urlencode(3) and khttp_urldecode(3), replacing the legacy kutil_urlencode and kutil_urldecode. They're identical except in how NULL values are handled, in the first case returning them as empty strings instead of NULL, in the second regarding them as errors. For the encoder, this allows all URL formatting tools to pass NULL values as query string values without errors. The earlier functions have been retained with the original behaviour with one exception, in that a NULL destination argument for kutil_urldecode triggers a KCGI_FORM return.

Have kxml_puts(3) and kxml_write(3) handle NULL pointers as content.

Verify and fix that all scope-opening functions in kcgixml(3) and kcgihtml(3) properly close variable arguments contexts on error.

Instead of aborting when the maximum number of scopes in kcgijson(3), kcgihtml(3), or kcgixml(3) has been reached, return KCGI_ENOMEM.

Check that the element index passed to kxml_pushnull(3) is valid.

Behaviour change: previously, several kcgixml(3) functions would return KCGI_FORM, such as when popping from an empty stack. To prevent other KCGI_FORM errors from being masked, use KCGI_WRITER to handle these situations.

Split apart kcgixml(3) into one manpage per function.

0.11.0

Kristaps Dzonsons

2020-03-27

Bumping minor number due to some API and behaviour changes.

API change: previously, khtml_ncr(3) accepted a uint16_t for its entity value. However, these values can legitimately be 32 bits. It has been changed to uint32_t.

API change: the struct khtmlreq pointer passed into khtml_elemat(3) is now const.

Behaviour change: previously, khtml_closeto(3) would return KCGI_FORM if given a stack position greater than the current stack. This is inconsistent with other functions, so such values are now simply ignored. It also masks other problems that cause KCGI_FORM to return. Furthermore, if this function was invoked at the current depth, it would close all scopes instead of none. This has also been fixed.

Behaviour change: previously, many kcgijson(3) functions would return KCGI_FORM if used out-of-context, for example, trying to open a named object in an array context. To prevent other KCGI_FORM errors from being masked, introduce a new error code KCGI_WRITER to handle these situations.

Passing a NULL pointer value to the string writing functions of khttp_puts(3), khttp_write(3), kcgihtml(3) or kcgijson(3) would cause undefined behaviour. Now these are noops.

Add khtml_printf(3). Split apart kcgihtml(3) into one manpage per function. Split apart kcgijson(3) into one manpage per function.

0.10.18

Kristaps Dzonsons

2020-03-22

0.10.16

Kristaps Dzonsons

2020-03-17

0.10.15

Kristaps Dzonsons

2020-03-11

0.10.14

Kristaps Dzonsons

2020-02-27

0.10.13

Kristaps Dzonsons

2020-02-17

0.10.12

Kristaps Dzonsons

2020-01-12

0.10.11

Kristaps Dzonsons

2019-07-05

0.10.10

Kristaps Dzonsons

2018-12-27

0.10.8

Kristaps Dzonsons

2018-12-18

0.10.7

Kristaps Dzonsons

2018-06-19

0.10.6

Kristaps Dzonsons

2018-04-19

0.10.5

Kristaps Dzonsons

2018-04-19

0.10.3

Kristaps Dzonsons

2018-04-10

The FastCGI release: when running kcgi's FastCGI mode on nginx, processes were being mysteriously killed under high load. This was due to the end-point closing the connection before all data was being read or written. To wit, I now establish a difference (in FastCGI) between the connection closing (which is a recoverable error) and the manager killing the connection or the control socket exiting, which are not recoverable. Since most of this development was on Linux/ARM with nginx, the sandbox for Linux has also been tooled up. A big thanks to Elouan Pignet, who was kind enough to diagnose the problem and provide access to his system for a fix, including several failed attempts. Thanks, Elouan!

To this end (API change), khttp_fcgi_parse(3) now returns the KCGI_EXIT when the system has exited. The KCGI_HUP is reserved for when the output channel has closed (after parsing) and the current connection is no longer valid. The documentation has been updated for relevant functions.

While studying these code paths, make sure that a sequence of writes (using khttp_write(3) or any of the writing front-ends) won't fail if khttp_body(3) wasn't able to complete due to the connection closing. Specifically, if the connection closes during khttp_body(3) (returning KCGI_HUP), the system will still expect headers. Earlier, it would assert with subsequent khttp_write(3) if the error were not caught and the In the modified behaviour, it will return KCGI_FORM to indicate that the system is out of state.

Make khttp_fcgi_parse(3) only require that callers invoke khttp_free(3) if exiting with success. This mirrors khttp_parse(3).

Merge a set of tutorial fixes from cyball, thanks!

Allow the kutil_log(3) functions to accept a NULL request. This makes it possible to use these functions for consistent logging without a request.

Lastly, run through all code snippets, apply style(9), and make sure that the MIME type is properly checked. And add a new tutorial, Best practises for pledge(2) security.

0.10.2

Kristaps Dzonsons

2018-03-24

0.10.1

Kristaps Dzonsons

2018-03-06

0.10.0

Kristaps Dzonsons

2018-01-18

A minor bump here due to API changes, along with a plethora of improvements: date handling, simplification of general code, and getting error codes exported in the write sequence. Many of the improvements and fixes in this release were implemented or suggested by Ingo Schwarze (schwarze@) in an audit generously funded by CAPEM Solutions, Inc. Thank you so much!

Memory leak fix: fix khttp_free(3) to free a forgotten allocation. Probably only affects FastCGI processes at 16 bytes lost per request parsed.

Feature: add the kcgi_writer_disable(3) function, which allows callers to determine which front-end writers (e.g., kcgixml(3)) are allowed to exist. This locks down the output formatting mechanism.

API change: the kcgixml(3) library now no longer prints the XML prologue with kxml_open(). This must be manually printed with kxml_prologue().

API change: the kcgijson(3), kcgihtml(3), and kcgixml(3) now return a consistent error code for all operations. The khtml_text function has been removed (it was deprecated).

API change: the kutil_urlencode(3) family of functions now all return NULL on memory failure. Earlier, this was inconsistent.

API change: the khttp_write(3) family of functions now all return enum kcgi_err to indicate a failure condition.

API change: the khttp_body(3) function now returns enum kcgi_err to indicate a failure condition instead of whether compression was enabled.

API change: the khttp_body_compress(3) function now returns enum kcgi_err to indicate a failure condition instead of whether compression was enabled. Furthermore, the comp argument simply dictates whether compression should be enabled or not, preventing confusion.

API change: the khttp_head(3) function now returns enum kcgi_err to indicate a failure condition. Furthermore, this function now dynamically allocates header lengths, removing prior bounds on header length.

While here, improve khttp_head(3), khttp_body(3), kvalid_string(3), and kmalloc(3) manpages.

Lastly, the kvalid_date(3) function has been generalised for arbitrary dates, prompting the addition of kutil_date_valid(3) and kutil_datetime_valid(3) functions to validate broken-down dates. And bring up to date with latest oconfigure.

0.9.10

Kristaps Dzonsons

2017-11-21

0.9.9

Kristaps Dzonsons

2017-10-30

0.9.8

Kristaps Dzonsons

2017-10-18

0.9.7

Kristaps Dzonsons

2017-09-23

Fix C headers in the documentation to be more minimal and standards-compliant.

Add Ross Richardson's Using Pages tutorial. Ross developed the new Undeadly Journal!

Bug fixes… Clarify some function elements in khttp_parse(3). Fixed undefined behaviour caused when khttp_templatex_buf(3) is passed a fallback function and mis-reports the input key length. (If you use templating with a fallback function, you must update.) Fix all instances of zero-length allocations. These are non-portable and might cause erroneous failure on some systems. Fix an error where using digest authentication via khttpdigest_validate(3) might crash with an unknown HTTP method. Most of these were found by Ingo Schwarze (schwarze@) in an extensive audit generously funded by CAPEM Solutions, Inc. Thank you!

0.9.6

Kristaps Dzonsons

2017-08-31

0.9.5

Kristaps Dzonsons

2017-05-31

0.9.4

Kristaps Dzonsons

2017-03-12

0.9.3

Kristaps Dzonsons

2017-01-25

0.5.9

Kristaps Dzonsons

2015-06-17

Add ability to retrieve HTTP headers from the request. This adds several fields to the structure filled in by khttp_parse(3) that allow for mapped (indexed by known header) and listed header fields. The HTTP headers are reconstituted from the CGI environmental variables according to the RFC. These fields existed in earlier releases, but were not documented and, moreover, did not reconstitute the HTTP header name from the CGI environment variable name. A regression test has been added to test this behaviour. While adding this documentation, clean up the type list in khttp_parse(3).

Add khttp_body_compress(3), which allows for stipulating auto-compression (the default for khttp_body(3)), compression without emitting the appropriate header (for applications providing the Content-Encoding parameter themselves), and no compression (for applications taking full control of output themselves). While there, make the test for requested compression be sensitive to the RFC 2616 qvalue. Both functions now return whether compression has been enabled. This functionality augments existing behaviour: it does not change it.

Add HTTP error codes described by RFC 6585.

0.4.2

Kristaps Dzonsons

2014-11-30

0.4.3

Kristaps Dzonsons

2014-12-03

0.4.4

Kristaps Dzonsons

2014-12-15

0.5

Kristaps Dzonsons

2014-12-18

0.5.1

Kristaps Dzonsons

2015-03-14

0.5.2

Kristaps Dzonsons

2015-04-05

Allow for opaque HTTP message bodies: if our content type isn't application/x-www-form-urlencoded, multipart/form-data, or text/plain (during a POST only), then accept the HTTP body as a single object and validate it against the empty-key validator. Add HTTP methods and headers stipulated by HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV) and Calendaring Extensions to WebDAV (CalDAV). Allow for the HTTP request headers to be exported to the struct kreq object as both a list and, for common HTTP headers, an indexed map. Added kcgixml(3) bits for some simple XML support and added khtml_putc() and khtml_puts() to kcgihtml(3) for consistency. Specify that a NULL template passed to the khttp_template(3) functions simply causes the named file or buffer to be outputted without any processing. Lastly, recognise getentropy(2) as a white-listed system call in the systrace(4) sandbox.

0.5.3

Kristaps Dzonsons

2015-04-10

0.5.4

Kristaps Dzonsons

2015-04-12

0.5.5

Kristaps Dzonsons

2015-04-12

0.5.6

Kristaps Dzonsons

2015-04-23

0.5.7

Kristaps Dzonsons

2015-04-27

0.5.8

Kristaps Dzonsons

2015-05-11

0.6

Kristaps Dzonsons

2015-07-07

0.6.1

Kristaps Dzonsons

2015-07-09

0.6.3

Kristaps Dzonsons

2015-07-18

More FastCGI: move control socket handling into its own process. The control socket is a UNIX socket (or similar) bound by the FastCGI runner, which is either the web server or an application like kfcgi(8), then passed as STDIN_FILENO to the FastCGI application. kcgi will then wait on this socket for incoming connections, which are acted upon with khttp_fcgi_parse(3). In this release, this logic has been moved into its own process instead of being managed by the web application itself during calls to khttp_fcgi_parse(3). While here, I cleaned up and simplified a lot of the sandbox and inter-process socket logic. The control socket is not yet sandboxed: that will come with later releases. Again, the FastCGI implementation is experimental!

Patches for deployment on FreeBSD contributed by Baptiste Daroussin—thanks! In analysis, found that the argfree function to khttp_parse(3) wasn't being invoked if the arg was itself NULL. (This is clearly bad behaviour—not all functions need that argument!) This has been fixed as well.

0.6.2

Kristaps Dzonsons

2015-07-15

This release includes an initial implementation of FastCGI. While the existing functionality wasn't logically altered, it was shuffled around quite a lot and abstracted to accomodate for the new FastCGI functions. The implementation is documented in kfcgi(8), khttp_fcgi_init(3), and khttp_fcgi_parse(3), and khttp_fcgi_free(3). This functionality is experimental, and needs strong analysis before casual deployment. It is also not quite feature-complete: HTTP compression is not yet functional for FastCGI applications. The kfcgi(8) launcher is also very bare-bones—this will be improved in subsequent releases. Again: this functionality is experimental and under development!

Also fixed a bug where calling khttp_free(3) after a prior khttp_parse(3) failure would cause a NULL dereference. Added the missing txt and xml suffixes to the suffix table. Fix that the request port number was erroneously disallowed to be >80.

0.6.4

Kristaps Dzonsons

2015-08-01

Add some proper security to kfcgi(8): place child FastCGI processes in a file-system jail, drop privileges, and ensure proper ownership of the socket. While here, begin sandbox mechanisms for the FastCGI control process. This only works for Mac OSX for now, but will expand to others. Again, until the next minor version bump, FastCGI support is experimental.

On that note, finalise the API for FastCGI calls. This involved moving most function arguments previously in khttp_fcgi_parse(3) into khttp_fcgi_init(3). To wit, the khttp_fcgi_parsex() function has been removed: all of the logic has been moved to the initialisation function, making the parse function much simpler. Cement this by adding several new regression tests that exercise the FastCGI functionality. These, of course, required that FastCGI functionality be added to the regression suite. This is documented in kcgiregress(3) (the manpage was renamed from kcgi_regress).

In adding the regression suite, one critical FastCGI bug was found (and fixed) for forms >1 kB. Fix another critical bug when writing large contiguous blocks, which would be silently truncated. (A regression test has been added for this.) The same would trigger a failure on Linux due to the sandbox: this has also been fixed. Internally, the behaviour of read and write sockets is consistently non-blocking. There has been a report of large compressed output failing on FreeBSD, but this has not yet been verified.

Lastly, incorporate a patch from Baptiste Daroussin (thanks!) for a tighter Capsicum sandbox for the parser process.

0.7.0

Kristaps Dzonsons

2015-08-07

Implement the FastCGI control process's sandbox for all supported operating systems. (The FastCGI control process is forked from each application process, and is responsible for accepting new FastCGI connections and passing the descriptors to the application for output and worker process for parsing.) This is the last feature for the FastCGI implementation: it now has the same protection as the CGI implementation for all child processes. Add sandbox for OpenBSD's tame(2), although this technically isn't supported yet (in snapshots, anyway) and returns ENOSYS. On OpenBSD machines with both sandboxes, this is tried first. This effort derives from a patch submitted by Reyk Floeter—thanks!

The FastCGI interface is no longer an experimental feature, but baked into the system.

In addition to the above, some variables were added to the manpages (which were also cleaned up a bit) and the AFL system was updated with the socket re-write.

0.7.1

Kristaps Dzonsons

2015-08-11

0.7.2

Kristaps Dzonsons

2015-09-04

0.7.3

Kristaps Dzonsons

2015-09-30

0.7.4

Kristaps Dzonsons

2015-11-12

0.7.5

Kristaps Dzonsons

2016-01-04

Added ability to easily perform HTTP digest authentication, khttpdigest_validate(3), which is heavily used in kcaldav and a few other systems. This manages all of the aspects of the sequences, from the child process performing an MD5 hash on the full data stream to the end check against a hash. While there, also add khttpbasic_validate(3) for completeness.

Add an experimental feature for debugging input and output streams. One can set debugging fields with khttp_parsex(3) or khttp_fcgi_initx(3) and have request reads or response writes logged to standard output. Note: this changes the system API, so make sure any systems calling the above functions are upgraded to account for the extra argument! Also merged a thorough documentation patch by Svyatoslav Mishyn—thanks!

0.7.6

Kristaps Dzonsons

2016-01-25

0.7.7

Kristaps Dzonsons

2016-02-10

0.7.8

Kristaps Dzonsons

2016-03-02

0.8.1

Kristaps Dzonsons

2016-03-15

0.8.2

Kristaps Dzonsons

2016-04-06

0.8.3

Kristaps Dzonsons

2016-04-19

Work around an old but fatal FastCGI problem only found on OpenBSD's httpd. This assumes that HTTP headers are only on the first FastCGI packet, which is not part of the standard. The workaround is to have HTTP headers buffer just like the HTTP body, which will cause multiple headers to lump into (hopefully) one packet. Headers are still flushed when the HTTP body begins, however, although I'm still unsure whether this is a good idea. (It pushes the status code to the web server faster, but incurs an extra trip on the wire.) If you've disabled buffering in khttp_fcgi_init(3), or you have so many headers that the output buffer is flushed before the last header, httpd will intermix your body with headers. Ew. This problem was raised as issues/2 by @cornett—thanks! Also fix issues/3, raised by the same—thanks again!

0.8.4

Kristaps Dzonsons

2016-06-14

0.8.5

Kristaps Dzonsons

2016-08-19

0.9.0

Kristaps Dzonsons

2016-10-10

0.9.1

Kristaps Dzonsons

2016-10-17

0.9.2

Kristaps Dzonsons

2016-11-27