hMailServer 6.3.3

Delivery

An outbound BDAT chunk larger than one send buffer no longer stalls (issue #261). The connection sends a message 60,000 bytes at a time; the SMTP client armed the read for the reply as soon as the BDAT command - or, with PIPELINING, the envelope - had been queued, and the second buffer of the chunk was then queued behind that read in the connection's operation queue, which starts only the operation at its head. The read could not complete while the remote waited for the bytes behind it, so nothing sent them, and the remote gave up after its own timeout (iCloud's 421 after five minutes, in the report). Every message over 60,000 bytes to a server advertising CHUNKING has stalled this way since outbound BDAT arrived in 6.2.28; a message that fit one buffer went, which is why short ones did; DATA never met it, because its body streams only after the 354. The operation queue runs once more after a read has started, so a write behind it goes at once. Two regression tests relay a 222 KB message over BDAT, with the envelope pipelined and without. Until you upgrade, OutboundChunking=0 in hMailServer.ini is the workaround.

The full-text indexer no longer reports an error for a message deleted before its terms were saved. On PostgreSQL the foreign key refuses the terms of a row that is gone (SQL Server Compact accepted them); the message is logged at debug level and skipped, since there is nothing left to index.

The webmail

Rebuilt to the shape of Gmail and Outlook.com. The page is real files now (Portal.html, Portal.js), embedded into the binary at build time. A top bar with one search box and its options panel; a navigation column with the system folders by their designation, then the account's folders, labels, Held mail, Contacts, Scheduled and the storage meter; a list whose row is a checkbox, a star, an avatar, the sender's name, the subject and the first line of the text, an attachment mark and a time, with archive, delete and mark-read under the pointer; a reading pane on the right, below, or off; a toast with Undo after every move; the keys a mail client has (c, r, a, f, s, o, u, /); settings behind the gear; the same page on a phone; light and dark.

Writing. The compose window docks at the bottom right, minimises to its bar or opens full; a reply, reply to all or forward is written under the message it answers, with the earlier messages of the conversation above it as cards. An attachment reminder asks once before a send whose text speaks of an attachment and carries none. A message, or the message being written, pops out into a window of its own.

The list. A row dragged onto a folder, with undo. A right-click menu on a row (open, archive, junk, delete, read, star, pin, block, sweep). A pin keeps a message at the top of its list (the $Pinned keyword, the way every mail program sees it). Block this sender writes a rule that files the sender into Junk; Delete all from this sender sweeps the folder. Shift-click selects a range and ctrl-click one row, as every desktop client has them. A selection can take in every message of the folder, or of the open tab.

Mute, and the inbox in tabs. A muted conversation leaves the inbox, its messages carry $Muted, and a rule the page writes files whatever answers any of them into the archive; Undo takes all of it back. The inbox comes in tabs - Primary, Social, Promotions, Updates and Forums the way Gmail sorts them, or Focused and Other the way Outlook does, or one list - the server saying which from the headers alone (the sender's domain, List-Post, List-Unsubscribe, Precedence, Auto-Submitted), the reader overruling it per sender from the row's menu.

Search. The last ten searches are offered under the empty box (kept in this browser, forgotten at sign-out), and a contact's name typed there completes to from:<address> from the address book. Fourteen more operators, parsed by the server beside the ones it had: cc:, bcc:, filename:, larger:, smaller:, older_than:, newer_than:, is:muted, is:pinned, category:, has:link, in_reply_to:, a -word to exclude, and OR. The reference is hmailserver/docs/WebmailShortcuts.md.

Follow-up, nudges, quick steps, clean-up. A flag can carry a due date - today, tomorrow, next week or a date - kept as $FollowUp and a $Due-YYYY-MM-DD keyword on the message itself, so other clients still see a flag and a move keeps the date; the row shows it, the Starred view sorts by it, and a reminder comes once a day. Nudges say "Received N days ago. Reply?" on a message that asked and "Sent N days ago. Follow up?" on one nothing answered; a preference turns them off. Quick steps are buttons a reader defines - move, mark read, label, forward, in one press, on the digit keys. Clean up conversation deletes the messages whose every line a later message quotes, keeping the newest, the unread, the starred and those with attachments, with Undo.

Everything above is in the twenty languages the page speaks, and the harness that executes the page's script runs 224 checks on it in CI.

CardDAV

The account's address book, on phones and desktop clients (RFC 6352). On the web services listener: /.well-known/carddav redirects to /dav/, a PROPFIND there answers the principal, the principal the address-book home, and the home holds one book, Contacts - the address book the webmail keeps and completes recipients from. The card a client sends is the card it reads back: the card itself is stored beside the contact (schema 6040), vCard 3.0 or 4.0, both advertised, and an edit made in the webmail is written into the stored card with the UID and the telephone number the phone gave it left alone. HTTP Basic as the account or an application password, over HTTPS only, with the IMAP logon's lockout and auto-ban; addressbook-query, addressbook-multiget and sync-collection reports; ETags with If-Match and If-None-Match. iOS, macOS Contacts, DAVx5 and Thunderbird speak it. The web services HTTPS listener has to be on (WebServicesHttpsPort).

The Linux control panel

Measured. build/check-deck-parity.py counts every field the desktop Control Panel writes against the REST API and the browser Control Deck, from the sources; hmailserver/docs/DeckParity.md is its report. Of 330 properties, 240 were writable over REST and 153 reachable from a Deck view at the start of 14 September; 328 and 322 by its end, the two not writable being groups and their members.

Moved. New write routes: the anti-virus settings as a group (/api/v1/settings/antivirus), sixteen more account fields and nine more domain fields on their PUTs, the distribution list's settings and a PUT for it, DNS blacklists, SURBL servers, white-list addresses, blocked senders, incoming relays, blocked attachments and the greylisting white list as resources, the cache group's eight ceilings and lives, an IP range's expiry, abort_spam_flagged on a rule's forward and reply, user_interface_language, and - under an account's address, for the administrator - its application passwords, its folders' permissions and its messages. The Deck gained a harness of its own (build/check-deck-script.py, 292 checks, in CI), full domain editing, an IP-ranges view with expiry, fetch-account and backup views, the account editor in full, distribution lists and aliases under their domain, the seven small collections, server messages, and the scripting, cache and indexing groups with their verbs.

The Linux regression suite

The route backlog, waves A to H. A REST route for every family the Linux fixtures had skipped: the directories and the ini section of the settings, the logon-failure list, domain aliases, a domain read whole and renamed, an IP range updated in place, the server messages, Sieve evaluation, fetch accounts, scripting, the backup settings, an account read whole, the caches, the message index and the relayer's password - and a build object on /api/v1/status. Then, in the day's last batch, the three families that stood behind most of what was left: application passwords administered under /api/v1/accounts/{address}/app-passwords, folder permissions under .../folders/{id}/permissions with the rights by the names the COM interface gives them and the letters SETACL takes, and the message object - a folder's whole live collection, a message read as a row with every header, a raw .eml added on APPEND's path, the flags changed on STORE's, a delete on EXPUNGE's - with the shims rewritten so that a fixture skips only against an older server whose OpenAPI lacks the route. The hosted Linux run on this release's tree: 1520 tests, 1289 passed, 0 failed, 231 skipped. Three server defects found on the way: a vacation save PostgreSQL refused, ncsa refused as a log format, a negative thread id on Linux. And one on every Windows installation, found by the first honest Windows gate of the branch: a domain's relay password could not be saved - the column was 255 characters and a DPAPI envelope is 314. Schema 6039 widens it.

The Control Panel

The live update from an open Control Panel. The helper ends whatever runs from under the installation before each installer run, the Control Panel closes itself once the helper has started, and a failed run's outcome quotes what the installer's own log says went wrong. See the note at the top.

Two defects the tidy branch's own gates found. generate-com-wrapper.ps1 died under Windows PowerShell the first time its MIDL path ran (MIDL's stderr became a terminating error); a COM error out of a recordset walk ended a scheduled task with a minidump and no description - ADORecordset::MoveNext catches it now (HM5037) and the exception handler names COM errors. The fetch-account editor's connection-security captions no longer carry POP3's port numbers into an IMAP account.

Security, from the release's own review

An adversarial review of everything the server gained since 6.3.2 ran before this release was cut, and what it confirmed is fixed here. The [Settings] section of hMailServer.ini over REST answered to any api key: a read-only key could read every secret the section holds (the OAuth2 HMAC secret, the password pepper, the service account's password), and a write key could set AutoBanCommand, which the firewall reconciliation runs as the service. The four ini routes answer to the administrator password only now, as the api-key routes do; a key of any scope gets the same 401. A key restricted to a domain could lift that domain's limits through PUT /api/v1/domains/{domain} - the eleven fields the COM interface gates on the server administrator - and is refused them with 403 now. In CardDAV, a REPORT body of a million ampersands cost a worker thread tens of seconds (the entity scan is bounded), a multiget answered the same href as often as it was asked, five thousand times at most (a repeated href is answered once, and a multistatus over 64 MiB is refused), and on MySQL the card column could not hold the megabyte the collection advertises (it can). Smaller: a refused save on a collection update no longer leaves the in-memory item changed; one restart thread at a time; the escaper's PostgreSQL flag is re-read per statement; the packaged fail2ban filter matches a logon failure whose name carries a double quote; an XML name may not carry an ampersand, a semicolon or a control byte. Two tests hold the two key-scope findings.

Build and supply chain

One header on every source file, the project's, in one form, put there and checked by build/add-license-headers.py.

The container image job of a tag run asks docker for the image by a lower-case name. At 6.3.2 the image was pushed and the smoke test that starts it was refused the capital letter in the owner's name; the image itself was fine.

The Linux suite's shims learned the anti-virus group and the nine domain properties the new fixtures read.

Database

On PostgreSQL, a backslash is stored as one backslash. Every value with a backslash in it - a Windows path in a DKIM key-file setting, a backslash in a signature, a vacation message or a rule - was stored with the backslash doubled, because the escaper doubled it for PostgreSQL unconditionally, which is right only while standard_conforming_strings is off, and it has been on by default since PostgreSQL 9.1. The connection now reads the server's own answer when it comes up and doubles only where the server would otherwise read the backslash as an escape. Found by the hosted Linux run reading a domain's key-file path back as C:\\keys\\next.pem. Values already stored doubled stay as they are; edit and save them once.

Schema 6039: a domain's relay-password column widened for a DPAPI envelope. Schema 6040: the CardDAV tables. Both upgrade in place.

Verification

The full regression suite ran on the stamped binary: 2,302 tests, 2,294 passed, 0 failed, 8 skipped. hMailServer.exe 6.3.3.42, SHA-256 2c57f78050ecc43ee6d69ac3a2dc42b1dd802cc35ec369c09457d8119d53510a.

The installer has two hashes, and both are given deliberately. hMailServer-6.3.3-x64.exe as built, 79,926,537 bytes: SHA-256 45a9d7f1a78bb31031dde27849fa21828a60aff178f32cbc2b0d9cd93f6b33f2. As published, after Authenticode signing: SHA-256 3188BE2FA16A4C43D3B43DDBD7274E4CFE878C082343EF25FFBD00489610A057.

The assertion build, first. A separate build with HM_ASSERT on - every internal invariant checked and any failure fatal - ran the whole suite before the shipping binary was built: 2,302 tests, 2,294 passed, 0 failed, 8 skipped. Nothing asserted.

Reproducible. Three clean Release builds of the stamped tree - the first two compared for the purpose, the third the one that ships - produced the same hMailServer.exe.

On the way here. Every batch that landed on master went through the full Windows suite on its exact tree before it was pushed, and the hosted Linux run on the last of them: the numbers are in the wiki's "Changes since 6.2.10" page, section 4d. The hosted Linux run on the release tree itself: 1520 tests, 1289 passed, 0 failed, 231 skipped (run 34904809203).