Skip to content
noggin

Core TLS: why the client handshake never completes, and what to try (2026-10-31)

Question: on the Core (ESP32-C3, vendor MicroPython 1.23.0 preview 2e7b6351 on 2026-03-25, _mpy 774), SSLContext(PROTOCOL_TLS_CLIENT).wrap_socket(sock, server_hostname=...) blocks for minutes, with or without settimeout, against example.com (by IP, CERT_NONE) and against Mosquitto 2.1.2 / OpenSSL 3 on the Mac (mutual TLS, EC P-256 DER certs, TLS 1.3 default and tls_version tlsv1.2). The broker logs “New connection”, then “disconnected: exceeded timeout” after 30 s. openssl s_client to the same broker works. Nothing was run on the board for this note.

Sources: upstream MicroPython pinned to tag v1.23.0 (MP/ = https://github.com/micropython/micropython/blob/v1.23.0/), ESP-IDF v5.2 (IDF/ = https://github.com/espressif/esp-idf/blob/v5.2/), Mosquitto v2.1.2 (MOSQ/ = https://github.com/eclipse-mosquitto/mosquitto/blob/v2.1.2/), MicroPython issues by number (https://github.com/micropython/micropython/issues/N). 2e7b6351 is not an upstream MicroPython commit (GitHub API 422), and Bambu’s repo (https://github.com/CyberBrick-Official/CyberBrick_Controller_Core) has only docs and app .py files, no firmware source or sdkconfig. So the vendor build is upstream-1.23-shaped by inference; every port detail below is upstream’s and needs the board readings in the last section to confirm.

Most likely: the ESP-IDF heap is starved, so Wi-Fi stops receiving mid-handshake. It is not TLS 1.3, not the certificate type, and not the broker.

  • mbedTLS on the esp32 port allocates from the IDF heap, not the Python heap (CONFIG_MBEDTLS_DEFAULT_MEM_ALLOC=y, MP/ports/esp32/boards/sdkconfig.base#L79-L86). It needs a 16 KiB receive buffer and a 4 KiB send buffer per connection (MBEDTLS_SSL_IN_CONTENT_LEN 16384, OUT 4096, IDF/components/mbedtls/Kconfig#L74-L99; MicroPython keeps the asymmetric split, sdkconfig.base#L59-L61). jimmo measured about 35 KiB for mbedTLS plus about 7 KiB for lwIP for one HTTPS request, on top of about 45.5 KiB for Wi-Fi (#8940).
  • The Python heap starts at 64 KiB on the C3 and grows into the IDF heap on demand: MICROPY_GC_INITIAL_HEAP_SIZE 64 KiB, MICROPY_GC_SPLIT_HEAP_AUTO 1 (MP/ports/esp32/mpconfigport.h#L29-L40, #L87-L88). Upstream sized it so the heap “can grow the heap once (double it) and still have enough internal RAM to start WiFi and make a HTTPS request” (same lines). Once memory has moved to the Python heap, it never goes back to the IDF heap (projectgus, #14421).
  • Our reading of 69 KB gc.mem_free is already more than the 64 KiB starting heap. So either the heap has grown (memory taken from the IDF) or Bambu raised the starting size. Either way, the IDF has less memory than upstream planned for. The C3 build also enables the BLE controller and NimBLE (MP/ports/esp32/boards/ESP32_GENERIC_C3/mpconfigboard.cmake pulls in sdkconfig.ble, which sets CONFIG_BT_ENABLED=y). Bambu’s frozen modules (ble_module, rc_module) may take more memory still (docs/research/2026-10-31-radio.md).
  • What starvation looks like on the wire: in #16650 (ESP32, open), tcpdump shows the ClientHello sent and the server’s flight ACKed. Then the device stops answering ARP for 50 s (“no ping responses”), while isconnected() stays True. Making the Python heap smaller fixed it. #14421 (an ESP32-C3 without PSRAM) is the same: once mem_info’s “max new split” fell to 248 bytes, Wi-Fi “stops working, but sta.isconnected() == True” and new sockets fail with ENOBUFS. Both fit our facts: TCP connects (so the broker sees “New connection”), then nothing.
  • It fails the same way with CERT_NONE against example.com as with mTLS against Mosquitto. That rules out certs, verification and the broker. It fails with tlsv1.2 too, and the IDF default builds no TLS 1.3 anyway (MBEDTLS_SSL_PROTO_TLS1_3 default n, IDF/components/mbedtls/Kconfig#L176-L180).
  • The broker’s 30 s says only “no MQTT CONNECT yet”. Mosquitto gives a new connection keepalive 20 s (MOSQ/src/context.c#L86) and drops it after keepalive × 1.5 (MOSQ/src/keepalive.c#L171-L172, #L203-L206), which logs “exceeded timeout” (MOSQ/src/loop.c#L332-L333). It does not show how far TLS got.

One thing the starvation story does not explain yet: settimeout should not hang for minutes. Upstream, a blocking socket with a timeout polls lwip_recvfrom in 100 ms steps and returns MP_ETIMEDOUT when time is up (MP/ports/esp32/modsocket.c#L565-L578, #L629-L661). That error is not a “non-blocking” one (EAGAIN/EWOULDBLOCK only, MP/py/stream.h#L133), so _mbedtls_ssl_recv passes it on (MP/extmod/modtls_mbedtls.c#L466-L480). The handshake loop then exits and raises OSError(116) (modtls_mbedtls.c#L524-L530, #L103-L119). If the board ignores the timeout, the wait is not in recv (it is stuck in C, or the MicroPython task gets no CPU), or the vendor fork differs. Also, after the broker closes the socket at 30 s, recv should return 0 and the handshake should fail at once (modsocket.c#L644-L652). Blocking past that means the Core no longer receives packets at all, which is again starvation (#16650).

Ruled out as the main cause: ECC speed. The C3 has a bignum (MPI) accelerator but no ECC accelerator (SOC_MPI_SUPPORTED 1 and no SOC_ECC_SUPPORTED in IDF/components/soc/esp32c3/include/soc/soc_caps.h#L56; MBEDTLS_HARDWARE_ECC depends on SOC_ECC_SUPPORTED, Kconfig#L465-L468). So P-256 runs in software. We found no primary C3 timing. But an ECDHE-ECDSA handshake is a handful of P-256 operations, and minutes would mean each one takes tens of seconds.

  • Supported IDF for v1.23: v5.0.4, v5.0.5, v5.1.2, v5.2.0 (MP/ports/esp32/README.md#L31). They ship Mbed TLS 3.4.1 (v5.0.4) to 3.5.0 (v5.2) (espressif/mbedtls include/mbedtls/build_info.h at the IDF submodule SHAs 727a037d and dd91f300). ssl.MBEDTLS_VERSION is MBEDTLS_VERSION_STRING_FULL (modtls_mbedtls.c#L87, #L763).
  • Curve25519 and the Brainpool curves are compiled out (sdkconfig.base#L69-L73), so P-256 is the cheapest ECDHE group. RSA key exchange is built in (MBEDTLS_KEY_EXCHANGE_RSA default y, Kconfig#L662-L664) and uses the MPI accelerator.
  • The buffer sizes are fixed when the firmware is built. MicroPython does not expose max-fragment-length or a dynamic-buffer option (MBEDTLS_DYNAMIC_BUFFER default n, Kconfig#L100-L105), and we never flash firmware (ADR 0002). So the fix has to be in our own memory use, not in mbedTLS settings.
  • SSLContext.set_ciphers/get_ciphers exist (vendor docs/library/ssl.rst#L60-L68; modtls_mbedtls.c#L305-L314). mem_info prints “max new split”, which is the largest free IDF block (MP/py/gc.c#L1181, MP/ports/esp32/gccollect.c#L83-L84). esp32.idf_heap_info(esp32.HEAP_DATA) lists each IDF region as (total, free, largest free block, min free) (MP/ports/esp32/modesp32.c#L193-L213; docs https://docs.micropython.org/en/v1.23.0/library/esp32.html#esp32.idf_heap_info).

Run each test from a fresh boot with our program stopped (raw REPL, mpremote over USB, so the Link is idle).

  1. Read memory and the build. Run import gc, micropython, esp32, ssl, machine; print(ssl.MBEDTLS_VERSION, machine.freq()); gc.collect(); micropython.mem_info(); print(esp32.idf_heap_info(esp32.HEAP_DATA)) once before Wi-Fi, once after connect, and once right before wrap_socket. Expected: Mbed TLS 3.4.x or 3.5.x and 160000000. If starvation is the cause, “GC: total” is above 65536 and “max new split” or the largest IDF block after Wi-Fi is under about 20 KiB, with the IDF’s total free well under the ~42 KiB that TLS plus lwIP need (#8940).
  2. Watch the wire from the Mac during one attempt: sudo tcpdump -ni en0 host <core-ip>, and ping <core-ip> in a second terminal. If the ClientHello goes out, the server’s flight is ACKed, then ping and ARP go silent: starvation, as in #16650. If no ClientHello is ever sent: stuck before the first write (memory allocation or the RNG). If ping still answers but the Core sends nothing: the CPU is stuck in mbedTLS.
  3. Run settimeout(5) then wrap_socket, and press Ctrl-C after 20 s if it is still blocked. Upstream code says this raises OSError(116) after about 5 s. If Ctrl-C breaks in with a traceback, the vendor fork’s recv loop differs. If Ctrl-C does nothing, the MicroPython task is stuck in C or gets no CPU, which matches test 2’s silence.
  4. TLS first, before anything else allocates. After reset: gc.collect(), connect Wi-Fi, then TLS at once (no BLE, ESP-NOW or our program loaded). Expected: the handshake finishes in seconds, and test 1 shows a larger IDF block. If it works here but not later, the fix is in our code: allocate less, allocate big buffers once at start so the Python heap never grows, keep one TLS socket open for the whole session, and set ctx up early.
  5. A smaller server flight. Use a single self-signed leaf with no chain, and RSA-2048 against EC P-256. With ctx.set_ciphers(["TLS-RSA-WITH-AES-128-GCM-SHA256"]) (check ctx.get_ciphers() first) the key exchange runs on the MPI accelerator with no ECDHE. Expected: no change if starvation is the cause, so this tells memory from CPU. It gets faster if the CPU was the cause. This cipher has no forward secrecy, which is fine for a test only.
  6. Pin the broker to TLS 1.2 with one suite (ciphers ECDHE-ECDSA-AES128-GCM-SHA256 in mosquitto.conf). This is already shown not to fix it, so it is low value: keep it only to shorten the test matrix.
  7. Longer timeouts do not help. The broker’s 30 s is fixed before CONNECT (keepalive 20 × 1.5), and a working handshake should take seconds. Raising the timeout only hides the hang.

If tests 1, 2 and 4 confirm starvation and test 4 still fails with the Python heap untouched, the vendor firmware leaves too little IDF RAM for TLS on this chip. Then mTLS for the Link is off the table without a firmware change. That is a decision for ADR 0002 and the standards audit (docs/research/2026-10-31-standards-audit.md, row 4), not a tuning problem.

Board results, 2026-11-01 (three attempts, #158)

Section titled “Board results, 2026-11-01 (three attempts, #158)”

The probe ran from boot.py before the eye program, logging to a file on the Core.

  • Wi-Fi joins in 3.7 s, and ntptime sets the clock.
  • At that point MicroPython’s GC heap has about 86 KB free, but the ESP-IDF system heap that mbedTLS allocates from has only 19,112 bytes free, in a largest block of 18,432. mbedTLS 3.5.0 needs about 35 KiB for one connection.
  • EC P-256 mutual TLS: TCP connects, then wrap_socket times out (OSError 116) and Mosquitto logs “exceeded timeout”. That is the starvation the research predicted.
  • RSA-2048: load_cert_chain refuses the key (ValueError('invalid key')), as PKCS#1 DER and as PKCS#8 DER.

Verdict: Bambu’s build reserves nearly all RAM for MicroPython’s heap, so no TLS connection fits beside Wi-Fi. A custom firmware with a smaller GC heap could change that, but that is out of scope. The Link stays on signed UDP (ADR 0005); MQTT (#157) is closed.

byYellow PineYellow Pine

noggin v0.2.26-6-g09bf9b7 · 09bf9b7