Abstract:This paper introduces multimodal conformal regression. Traditionally confined to scenarios with solely numerical input features, conformal prediction is now extended to multimodal contexts through our methodology, which harnesses internal features from complex neural network architectures processing images and unstructured text. Our findings highlight the potential for internal neural network features, extracted from convergence points where multimodal information is combined, to be used by conformal prediction to construct prediction intervals (PIs). This capability paves new paths for deploying conformal prediction in domains abundant with multimodal data, enabling a broader range of problems to benefit from guaranteed distribution-free uncertainty quantification.
| Comments: | Code available at this https URL 20 pages, 34 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2410.19653 [cs.LG] |
| (or arXiv:2410.19653v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2410.19653 arXiv-issued DOI via DataCite |
Submission history
From: Alexis Bose [view email]
[v1]
Fri, 25 Oct 2024 15:56:39 UTC (38,258 KB)
[v2]
Mon, 28 Oct 2024 14:48:50 UTC (38,258 KB)
[v3]
Thu, 14 May 2026 17:27:52 UTC (12,118 KB)