In indoor navigation tasks, the use of floor plans is an efficient and cheap way to provide globally consistent metric and topological information about various environments. However, most studies on floor-plan-based navigation have relied on LiDAR rather than RGB cameras because of the difficulty of performing cross-modality matching. In this paper, we instead focus on the visual indoor navigation problem and propose VF-Nav, a visual floor-plan-based point-goal navigation algorithm combining a brain-inspired localization method with a topological planning technique. In the proposed approach,continuous and accurate localization is achieved by combining the metric information provided by the floor plan with a brain-inspired localization model. Then, the global path to the point goal is generated by building the topological map from the floor plan, and a short-term target is provided at each step. Finally, a reinforcement learning control module guides the robot to reach each short-term target. The experimental results on a simulated point-goal navigation dataset demonstrate the excellent performance of the proposed approach in a complicated indoor environment. Our method achieves a success rate of up to 88% and a success weighted by path length of 71%.