-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
Resolve type instability issues in aggregation #2424
Comments
In the following functions, the first two allocate, but not the last one: function fill_row2!(row, col::AbstractVector,
i::Integer,
colnames::NTuple{N, Symbol}) where N
@inbounds for j in 1:length(colnames)
cn = colnames[j]
val = row[cn]
col[i] = val
end
return nothing
end
function fill_row2b!(row, col::AbstractVector,
i::Integer,
colnames::NTuple{N, Symbol}) where N
@inbounds for cn in colnames
val = row[cn]
col[i] = val
end
return nothing
end
function fill_row3!(row, col::AbstractVector,
i::Integer,
colname::Symbol) where N
val = row[colname]
col[i] = val
return nothing
end A possible workaround is to require column names to be always in the same order: then we could index using an integer, which I just checked doesn't allocate. |
This is a bug. We have to fix it and insist that column names always have the same order. I will fix it. Otherwise we have:
so it is hard to predict the column order (it will depend on the first row processed, which in threading context is problematic) |
Also this is what we require in other places:
|
See https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3 and in particular
https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3#file-splitapplycombine-jl-mem-L1402
and
https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3#file-splitapplycombine-jl-mem-L1438
and
https://gist.github.com/nalimilan/b801d48ab2931b6a671f05081fd94ba3#file-splitapplycombine-jl-mem-L1439
The text was updated successfully, but these errors were encountered: