All of these set the font-size on the SVG element itself. The easiest way to override it globally is to add a cell with the following content to your notebook:
html`<style>svg { font-size: 12px !important }`
Alternatively you can modify the style property on the element before displaying it:
chart.style.fontSize = '12px', chart
A third option would be to wrap the chart:
html`<div class="custom">${chart}<style>
.custom svg { font-size: 12px !important }
`