Visio's file format, .vsdx, is the first thing to let go of, because none of the open source options here is built to read it. That sounds worse than it is. What they replace is the work, not the container: LibreCAD and QCAD speak DXF and DWG, the interchange formats that keep scaled drawings moving between CAD tools, while Heimer, VYM, and DesktopNaotu keep node-and-branch maps in their own simple XML or km files. Decide up front whether your Visio library is really technical drawing or really idea structuring, because that split maps almost exactly onto these two groups of tools.
The clean matches are the technical ones. If your Visio work is floor plans, elevations, or network layouts drawn to scale, LibreCAD reads and writes DXF and DWG directly and QCAD adds a scripting interface for repetitive drawings, so precise geometry survives the move. Flowcharts, org charts, and mind maps are a looser fit: the mapping tools here produce trees and connected nodes rather than Visio's free-form connectors and stencils, so expect to rebuild diagrams by hand rather than import them. Custom stencils, shape data, and data-linked diagrams do not carry across at all.
Keep your old diagrams readable while you move. Export the drawings you still need as PDF or SVG so they stay legible regardless of which tool wins, then rebuild only the diagrams that are still living documents. LibreCAD can batch-convert DXF files to PDF, PNG, or SVG from the command line, which helps when you have a large archive to publish rather than edit.