Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Connections: apply synapsesCompetition() in adaptSegment() #584

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Connections: apply synapsesCompetition() in adaptSegment() #584

wants to merge 10 commits into from

Conversation

breznak
Copy link
Member

@breznak breznak commented Jul 24, 2019 •
edited
Loading

  • SP uses synapse pruning, same as TM has been doing
    • make pruning default to true in adaptSegment (unless it breaks too many tests)?
      • or altogether remove the param and just use it?
    • this brings a performance gain about 5%
  • WIP Connections.adaptSegment calls newly added synapsesCompetition
    • results in more synapses being pruned -> which turns to another 5-10% speedup
    • better synaptic activations (fewer extremes) as described in the original PR
    • find way to set reasonable default paramaters
  • tests expected to fail with mismatches

Follow up to #466
Fixes #558

breznak added 5 commits July 24, 2019 01:05
@breznak
Connections:destroySynapse fixup, use find
7394e27 
instead of lower_bound, caused err in Debug.
Seems faster
@breznak
SP: allow synapse pruning in adaptSegment
37a0d98 
already used in TM
makes SP 10% faster! (on MNIST 55->50s)
@breznak
Connections:adaptSegment use synapsesCompetition
fd58f25 
normally in learning.
@breznak
Connections: introduce Random rng_ member
9a1bbf9 
and use it for synapsesCompetition()
@breznak
Revert "Connections: introduce Random rng_ member"
b2cc6c4 
This reverts commit 9a1bbf9.
@breznak breznak added SP optimization code code enhancement, optimization, cleanup..programmer stuff labels Jul 24, 2019
@breznak breznak self-assigned this Jul 24, 2019
@breznak breznak closed this Jul 24, 2019
@breznak breznak reopened this Jul 24, 2019
@breznak breznak closed this Jul 24, 2019
@breznak breznak reopened this Jul 24, 2019
breznak
breznak commented Jul 24, 2019
View reviewed changes
src/htm/algorithms/Connections.cpp Outdated
@@ -479,6 +475,9 @@ void Connections::adaptSegment(const Segment segment,
}
}

//balance synapses using competition on dendrite
synapseCompetition(segment, 4, 10); //FIXME derive these numbers
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calling synapseCompetition from adaptSegment

  • this works pretty well, with synapse pruning many synapses get removed which leads to a considerable speedup
  • set the min/max parameters somehow smarter
    • I guess those should ideally be around stimulusThreshold ?
    • the tighter the boundary, the more syns get removed, the faster SP (on MNIST) is.
      • but where is the line? What is the disadvantage? I'd think too tight boundary is more prone to noice?

CC @ctrl-z-9000-times

src/htm/algorithms/Connections.hpp
@@ -510,7 +510,7 @@ class Connections : public Serializable
*/
void synapseCompetition( const Segment segment,
const SynapseIdx minimumSynapses,
const SynapseIdx maximumSynapses);
const SynapseIdx maximumSynapses);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO

  • this should have some parametric default values
  • replace (or provide overload) with percentages off the range [stimulusThreshold, maxSynapsesPerSegment]?

src/htm/algorithms/SpatialPooler.cpp
@@ -702,7 +702,7 @@ Real SpatialPooler::avgConnectedSpanForColumnND_(UInt column) const {
void SpatialPooler::adaptSynapses_(const SDR &input,
const SDR &active) {
for(const auto &column : active.getSparse()) {
connections_.adaptSegment(column, input, synPermActiveInc_, synPermInactiveDec_);
connections_.adaptSegment(column, input, synPermActiveInc_, synPermInactiveDec_, true);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SP also does prune synapses (like TM)

  • default to true/or even rm the parameter, when all use true now

@breznak breznak mentioned this pull request Jul 24, 2019
Open
2 tasks
@breznak breznak added the high label Aug 1, 2019
@breznak breznak mentioned this pull request Aug 1, 2019
Open
@breznak
Copy link
Member Author

breznak commented Aug 1, 2019

This PR now gains importance with a problem I've discovered for NAB detector usingour HTM: htm-community/NAB#15 (comment)

For this PR to be considered complete, we need to resolve "how do we choose the lower/upper bound for syn competition". Currently it is hardcoded. #584 (review)
Unless a better solution is found, I like the proposal: #584 (comment)
to provide percentages off the range [stimThreshold, maxSynapsesPerSegment], which would look like [stimThresh*(1.0 + lowerPct), maxSperSeg*(1.0 - upperPct)].

breznak
breznak commented Jun 4, 2020
View reviewed changes
Copy link
Member Author

@breznak breznak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aiming to finish this PR soon..

src/htm/algorithms/Connections.cpp
@@ -519,6 +519,9 @@ void Connections::adaptSegment(const Segment segment,
}
}

//balance synapses using competition on dendrite
synapseCompetition(segment, 8, 11); //FIXME derive these numbers
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the only big remaining issue - how to choose good values for these?
Any tips @ctrl-z-9000-times ?

# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@ctrl-z-9000-times ctrl-z-9000-times Awaiting requested review from ctrl-z-9000-times

@dkeeney dkeeney Awaiting requested review from dkeeney

At least 1 approving review is required to merge this pull request.

Assignees

@breznak breznak

Labels
code code enhancement, optimization, cleanup..programmer stuff high optimization SP
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

SP, TM use synapseCompetition()
1 participant
@breznak