Pretty sure you’ve encountered a bug. Logically, “non-allocating” would mean “you can’t write to it” and that appears to be the case from grepping through the zfs codebase:
if (txg == 0)
spa_config_enter(spa, SCL_ALLOC, FTAG, RW_WRITER);/*
* If the vdev is marked as non-allocating then don’t
* activate the metaslabs since we want to ensure that
* no allocations are performed on this device.
/
if (vd->vdev_noalloc) {
/ track non-allocating vdev space */
spa->spa_nonallocating_dspace += spa_deflate(spa) ?
vd->vdev_stat.vs_dspace : vd->vdev_stat.vs_space;
} else if (!expanding) {
metaslab_group_activate(vd->vdev_mg);
if (vd->vdev_log_mg != NULL)
metaslab_group_activate(vd->vdev_log_mg);
}
From this, it certainly appears to follow as though your pool has erroneously marked every vdev as non-allocatable, with uncertain effects. I’d strongly advise opening a bug report (and ideally, linking it here afterwards).