Project Design: Self Guidance
This task can be broken down into two fundamental questions:
- When do we change topics? (Function 'F ')
- What do we change to? (Function 'S ')
Ideally, this will break down to something of a formula, like in the attached picture.
*Note, func S will start as random selection, either to be picked up or dropped by func F.
How long will humans spend with a text, given its percieved difficulty/their familiarity with it? This piece of function F will be one of the main driving factors of the model's agency. It helps answer when we switch by assessing perplexity. It takes the absolute difference between some ideal perplexity p* and the actual perlexity measurement, where the closer that value is to 0, the more likely it is we'll continue reading from that document. If the actual measured perplexity is very high, then the document is understood to be too difficult. If it is very low, then we expect to know it well enough that it doesn't interest us. In either case, a high absolute difference is generated and we're more likely to stop.
Model's willingness to accept contradictory information/ rewrite old information. This part of func F can be thought of as some α(gradients) where 1. we find all gradients from timestep+1 that exceed a threshhold, and 2. of those, which are moving directly opposite to direction at the current timestep? Those could indicate direct opposing belief sets. Also, consider using an entailment model that takes in a sentence start and the model's prediction of and actual end of the end of that sentence. Compare the prediciton to the ground truth and get a measure (classification?) of agreement; thus, contradiciton can be identified. This will require opening another can of worms though, i.e. training up another model.
This piece of function S will might help us know what to search next as we find items of disproportionately high perplexity in relation to the text average. We might imagine a stack of max size n where we add new search topics. If we finish with topic as defined by function F, then we pop it from the stack and continue training on the preceding data. If we pass n, the bottom of the stack drops out. We could alternatively just let the F decide what to prune as it continues back down the stack of no defined capacity, or a combination of the two.
What information do humans intake and retain? Beginning and end of discourse? Titles and headers? Can maybe be implemented in search engine, as opposed to the model.
Given exposure to 100% of the text, what percentage do humans retain, and which pieces?
... we don't really want to touch this atm.
- Is the first fundamental question just a reinvention of curriculum learning?
- Is the second fundamental question just a reinvention of recommender systems? These don't inalidate our study, it may just provide us resources to turn to. Specificaly, look into SPCL.
- Do CL methods use all the training data, or do some draw from large coproa and just continue until they've reached an acceptable minima?