Canonicalized filepaths has a large performance impact on compilation speed
#546 opened on Feb 20, 2023
Repository metrics
- Stars
- (346 stars)
- PR merge metrics
- (PR metrics pending)
Description
Currently the compiler supports emiting "canonical" paths for modules, instead of say absolute, project file relative, etc, a list of possibilities can be seen in compiler/front/in_options.FilenameOption. Most pressingly, foCanonical results in massive performance problems as each attempt at canonicalzing a path results in a lot of filesystem IO.
The goal is fairly simple:
- get the baseline performance of compiling with
--filenames=canonicaland without - track down the various use cases and classify them by purpose/intention
- with the benchmark data and use cases in hand, eliminate/reduce the filesystem access (AKA cache the data)
To measure the impact search the /tests directory for tests that specify the --filenames=canonical option, then run those with and without to observe the difference. The largest impact was observed with ./koch.py temp --stacktrace --stacktracemsgs -d:nimCompilerStacktraceHints --lib:lib -d:debug -d:usenodeids c --filenames=canonical --msgFormat=sexp tests/compilerfeatures/tstructured_parse_fail.nim, as this resulted in setMsgFrame calls which converted TLineInfo to strings triggering the canonicalization for many hot compiler procedures.
If stuck at any point, ask @saem.