Reduce dependent functions

[mitchute]

cpgfunction is relying on some dependant functions that could possibly be eliminated.

• nlohmann json - cpgfunction is currently configured as a library, so this dependency should not be needed in the library portion of the code. If it's to be used as a standalone application, then you will need some sort of input processing. We should probably make the distinction between library and application more clear and getting this dependency in the right place will help with that. Beyond that, the input structure in the input JSON files appear to be like this: {"x": [1, 2, 3, 4], "y": [1, 2, 3, 4]}. This is basically just csv data. If that's all the input is, we could probably just move to a CSV input format and not have to carry around a separate copy of nlohmann json. That would be faster and eliminate this dependency.

• dcopy_ - is this just copying one vector to another? This could be done with standard C++ code. If you want to roll your own version, called "dcopy", fine, but if that's all it is, it would be better to write your own.

• dot - this looks like a dot product, but I'm not seeing it used. You have a _dot_product lambda function in heat_transfer.cpp, but that doesn't use the LA function. If it's not used, it should be eliminated.

• gemv - is this used outside of a test? I don't see it in the cpgfunction core source code.

I might update with others as I get further into the code.

[j-c-cook]

I have moved the BLAS and LAPACK functions to a LinearAlgebra repository. I think it makes the most sense to have that be separate from cpgfunction, and define it in third_party. My plan is to remove all BLAS and LAPACK calls to work through the LinearAlgebra library.

• on dcopy - if std::vector::copy is as fast as BLAS::copy, then I agree there is no reason to use BLAS
• on _dot_product in heat_transfer.cpp - that part of the code is outdated and currently contains a bug (see issue 10). I plan to remove _dot_product and make use of BLAS::dot; it is faster than multi-threading
• gemv is currently unused in cpgfunction, however, at one point I thought I may use it. I wouldn't mind if LinearAlgebra has more API references to BLAS and LAPACK than I am using. I think the testing of third_party libraries should be turned off when building cpgfunction.

[Myoldmopar]

Yeah, that LinearAlgebra submodule tripped me up. First I forgot to initialize and recursively pull the submodule, then I was surprised that it even existed. It appears to just be a small set of forwarding calls out to BLAS/LAPACK. But then in the cpgfunction code some of the forwarding calls are declared again, but in a different namespace, thus bypassing this library entirely?

My hope here is that we can actually trim down the dependency list, not add more API references to BLAS and LAPACK. I really don't want to have to add more Fortran code to the repo, and I would love to not have to install yet another linear algebra library. But something has to happen.

It would be great if we could get the number of calls to BLAS/LAPACK/Boost/Nlohmann trimmed down as far as possible, and then look at alternatives to the few remaining things we need. That would absolutely maximize the usefulness of this library, well beyond our own consumption with EnergyPlus. If you want other people to adopt it later, then you need to lower the burden of using it. If this could just include Eigen as a third party library in the repo and it is quickly all built in one pure C++ project, then this becomes super easy and usable in other modern projects.

[j-c-cook]

The reason they were declared again is because that version of LinearAlgebra assumed I would be passing a pointer to the beginning of the vector (std:vector.begin()). However, that is not always the case if you want the best performance. I had planned on making LinearAlgebra able to easily declare a different starting location by adding an overloaded function with additional integer arguments (and remove those forwarding calls from cpgfunction). The purpose of LinearAlgebra is so that it can be used in the future for other projects, and have tests and examples that are independent of cpgfunction.

I have been removing Boost dependencies by adding BLAS calls. The calls to BLAS vectorized function are much faster, and it even has a function very much niche to this project (spmv can perform matrix vector multiplication of a packed symmetric matrix). I was working towards ridding cpgfunction of Boost by first making use of all the linear algebra I can (rather than multi-threading), then trying to obtain similar speed performance in multi-threading from OpenMP for the computation of segment responses. Following that course of action, we would be left with implementing a Gaussian Quadrature Kronrod integration for the finite line source.

I like BLAS and LAPACK because you can declare these 1D vector's and essentially slice by giving a starting position and a number of elements to perform on (like here where I start at begin_q of q_reconstructed). It is a way to reduce the number of container declarations, which helps with speed and keeps memory low.

Perhaps these techniques I am saying can be done in Eigen with the same speed, same number of allocations and same consumption of memory. However, there is a possible path forward to remove the Fortran depencendy introduced by BLAS. The combination of CBLAS and LAPACKE would make this library dependent on C.

---

TLDR; Boost calls are being reduced by increasing BLAS calls. Boost dependency could be nullified. BLAS/LAPACK appear to be able to provide higher performance by being low level and minimal. IMO Eigen appears to be higher level than practical, and a bit convoluted.

[j-c-cook]

I’m going to look into removing the FORTRAN dependency this weekend by running some timing tests using BLAS++ and LAPACK++.

If that works it would hit two birds with one stone:

1. We could get rid of FORTRAN and just use C++
2. I could continue along the same lines of development I am on (with the intention of removing the boost dependency)

[mitchute]

Here's a couple of interesting notes regarding this thread:

1. A proposed PR to Eigen for integrating Gauss-Konrad algorithms, which was ultimately declined. http://manao.inria.fr/eigen_tmp/pullrequests/109/bb_dump.html

2. Using BLAS/LAPACK from Eigen.
   https://eigen.tuxfamily.org/dox/TopicUsingBlasLapack.html

[j-c-cook]

1. There are some other libraries that have the implementation, for example ALGIB has Gauss-Kronrod quadrature. Though I am not sure how large ALGIB is and whether or not we would want our own standalone implementation.
2. I have seen that before, and at that link it says "In order to use external BLAS and/or LAPACK library you must link your own application to the respective library and their dependencies." Therefore, we would still need the user to have BLAS/LAPACK installed on their system.