Personal Knowledge Databases

Tools For Reference and Learning

Daniel G. Martinez

Science Applications International Corporation

Reading as Distillation

We seek idiosyncratic knowledge

Our memory of interesting – even vital – facts and concepts that we read tend to dwindle so that we can't recall them at will.  And each such fact or concept that is interesting or vital to one of us is apt to be irrelevant to another reader of the same text.  This is to say that when we read, we distill text.  The tools described herein provide ways to facilitate the accumulation and retrieval of our conceptual distillates.  These tools transcend all disciplinary boundaries.

Annotation of reading materials

To rapidly assimilate the facts and concepts that one notices in printed materials, bird's wings (< word > ) are put around words that are to be included in data bases called a dictionary and thesaurus, and marginal tick marks signify a fact or concept that is to be included in another data base called a reader's index.  After a reading session data bases can quickly be updated.  And the very actions of annotating reading materials, and of systematically accumulating words, facts and concepts from them in manually-updated databases, is certain to boost the quality and quantity of one's learning.

Foundation Resources

Dictionary and Thesaurus

The ability to wield language well is a mainstay of scholarship, and building one's own dictionary and thesaurus can sustain a cycle of language development and reward that fuels itself.  It is not enough to use an authoritative dictionary: simple use of the Oxford English Dictionary would satisfy that requirement.  Instead, one must construct an idiosyncratic database of headwords, definitions and synonyms of one's own choosing for such a reference to be of greater utility than ordinary references.

Reader's Index

The ability to assimilate written material is essential to one's functioning in many spheres of endeavor.  For this reason we must be able to read at a fast rate, to discern relevant facts and concepts in what is read, and to recall that information on demand.  Too often, we are frustrated by our inability to recall the details of an important fact or concept that had been read but that can't even be associated with a particular book or author.  A properly maintained reader's index can allay such frustrations.

Dictionary and Thesaurus

Structure of Data Set

ω = {
{"headword 1","definition","synonyms"},{"headword 2","definition","synonyms"}, ... ,{"headword n","definition","synonyms"}
};

Ordinary Search & Report Function ( dictionary 1 )

\(p[t_] := 
      TableForm[
        Sort[\(Select[ω, StringTake[#\_\(\([[\)\(1\)\(]]\)\), StringLength[ToUpperCase[ToString[t]]]] == ToUpperCase[ToString[t]] &]\)\_\(\([[\)\(All, {1, 2}\)\(]]\)\)], TableSpacing -> {1}]\)

Ordinary Search & Report Function ( dictionary 2 & thesaurus )

g[t_, λ_, e_]:=Module[{τ = t, z = λ, 
            y = e, Ω, Ψ},    
\(Ω = 
            Table[
Length[
Flatten[
StringCases[
StringInsert[
ToString[τ\_\(\([[\)\(j, e\)\(]]\)\)], "_", \(-1\)], z ~~ _, 
                    IgnoreCase -> 
                      True]
]], 
{j, 
                Length[τ]}]; 

Ψ = 
            Table[
If[Ω\_\(\([[\)\(j\)\(]]\)\) > 
                  0, τ\_\(\([[\)\(j\)\(]]\)\)], 
{j, 
                Length[Ω]}]; 

Select[
Ψ, 
Length[#] > 0 &]\)
]

Reader's Index

Structure of Data Set

Document Function

β[x_]:=Switch[x,

1,"Stephen Hall  Invisible Frontiers  The Race To Synthesize A Human Gene",
2,"E. C. Pielou  The Energy Of Nature"]

Individual Entries by document and page number

κ={
{2, 17, "air temperature v. altitude"},
{2,0,"E. C. Pielou  The Energy Of Nature"},

{1, 
              4, "beef and pork pancreases supplied theraputic insulin to \
Americans in 1976"},
{1,0,"Stephen Hall  Invisible Frontiers  The Race To Synthesize A Human Gene"}
};

Search & Report Module

ri[x_]:=Module[  (*  Reader' s Index module, which uses module g to search through the third column in the κ list for substring x, and reports the results.  *)
\({ex = x, v, cc, s}, 
v = 
          g[κ, 
ToString[
              ex], 
3]; 
cc = 
          Complement[
            v\_\(\([[\)\(All, 1\)\(]]\)\)]; 
s[
            k_] := {β[
              k], 
TableForm[
Sort[\(Select[v, #\_\(\([[\)\(1\)\(]]\)\) == k &]\)\_\(\([[\)\(All, {2, 3}\)\(]]\)\)]]}; 
Do[
Print[
TableForm[
\(Table[s[cc\_\(\([[\)\(j\)\(]]\)\)], {j, 1, Length[cc]}]\)\_\(\([[\)\(j\)\(]]\)\)]], 
{j, 1, Length[cc]}]\)
]

Example of Reader's Index Query and Report

ri[
"beer"
]

Starter Kits

Recommendation is made that the Dictionary and Thesaurus be put into one file and the Reader's Index into another because of redundancies in their search and report functions, although they could be accommodated into one.  However, data set size might soon become unwieldy for most computers if they are kept in one file.

Dictionary and Thesaurus

Data Set

To enter a new headword, definition and synonym list, put the cursor to the left of the topmost headword then click on the button above the list.  Fill in the headword, its definition, and a list of synonyms ( if any ) in the style of the examples shown.  When list w1 has about 500 headwords, create list w2 above it and insert w1 into the Join command argument, viz.:  ω = Join[ w1, w2 ].  Continue in this fashion with as many lists as you want or as many as your computer will accommodate.  A suggestion is to load separate lists with their own headword categories, such as geographic, biographic, astronomic and so forth.

({{{"\[Placeholder]", "\[Placeholder]", "\
\[Placeholder]"},}})

\(\(w1 = {
{"TOY PROBLEM", "a simple illustration of how an algorithm works
an unrealistically easy proposition", "foil, example"}, {"IDIOSYNCRASY", "a structural or behavioral characteristic peculiar to an individual", "quirk, trait, characteristic, peculiarity"}, {"DWINDLE", "to become gradually less until little remains", "fade, decrease, diminish, subside"}, {"UNINTERESTED", "without an interest, such as financial", "apathetic"}, {"DISINTERESTED", "free of bias or self-interest", "impartial, fair"}, {"ENNUI", "listlessness and dissatisfaction resulting from lack of interest", "boredom, uninterested"}, {"ENMITY", "deep-seated, often mutual hatred", "hatred, animosity, malice, rancor"}, {"HEADWORD", "a word set in distinctive type that is a heading for an entry in a reference work", ""}, {"RELUCTANT", "exhibiting or marked by unwillingness", "disinclined, unwilling, averse, hesitant, demurring, diffident"}, {"BOISTEROUS", "noisy and lacking in restraint or discipline", "loud, noisy, uproarious"}, {"MILD", "gentle or kind in disposition or behavior", "meek, easygoing, patient, kind, gentle"}, {"PLEAD", "to appeal earnestly", "beg, implore, beseech, solicit"}, {"MEDDLE", "to interfere in the affairs of others", "interfere, obtrude, interlope, intervene, pry, snoop, impose, infringe"}, {"ENTICE", "to attract by arousing hope or desire", "lure, allure, attract, sweet talk, cajole"}, {"HOWEVER", "in whatever manner or way", "still, yet, nevertheless"}
};\)\)
ω = Join[w1];

Search & Report Functions

\(p[t_] := 
        TableForm[
          Sort[\(Select[ω, StringTake[#\_\(\([[\)\(1\)\(]]\)\), StringLength[ToUpperCase[ToString[t]]]] == ToUpperCase[ToString[t]] &]\)\_\(\([[\)\(All, {1, 2}\)\(]]\)\)], TableSpacing -> {1}]\)      
Off[
      General::"spell1"]
Off[
      StringTake::"take"]

g[t_, λ_, e_]:=Module[{τ = t, z = λ, 
            y = e, Ω, Ψ},    
\(Ω = 
            Table[
Length[
Flatten[
StringCases[
StringInsert[
ToString[τ\_\(\([[\)\(j, e\)\(]]\)\)], "_", \(-1\)], z ~~ _, 
                    IgnoreCase -> 
                      True]
]], 
{j, 
                Length[τ]}]; 

Ψ = 
            Table[
If[Ω\_\(\([[\)\(j\)\(]]\)\) > 
                  0, τ\_\(\([[\)\(j\)\(]]\)\)], 
{j, 
                Length[Ω]}]; 

Select[
Ψ, 
Length[#] > 0 &]\)
]

Search & Report Templates

p[en]                                      (*  dictionary 1 :  
            match first consecutive letters in headword and print headword and its definition  *)

(Subscript[(g[ω, 
"ter"
, 
            1]), [[All, 
          1]]])//TableForm  (*  dictionary 2 :  
            match substring in headword and print list of matches  *)

(Subscript[(g[ω, 
"trai"
, 
            3]), [[All, {1, 
            3}]]])//TableForm  (*  thesaurus :  
            match substring in synonym list and print headword and its synonyms  *)

Test Function

To take the test load the Data Set then select the closed cell below and press 'keyboard enter'.  Default test delay period is 4 seconds, inter-word period is 5 seconds, and 3 trials are presented per press.  To modify the time delays, open the closed cell and edit the Pause functions' arguments.

j = 0; 
    Do[b = (Subscript[ω, [[Random[Integer, {1, Length[ω]}]]]]); 
Print[
          (Subscript[b, [[1]]])]
Pause[4]
Print[
          (Subscript[b, [[2]]])]
Pause[5], 
{3}]

Single Sublist Viewer

After the Data Set is entered, select the closed cell below and press 'keyboard enter' to view arbitrarily-selected headwords, their definitions and their synonyms ( if any ).

nb = SelectedNotebook[];
λ = (Subscript[ω, [[Random[Integer, {1, Length[ω]}]]]]);
(Subscript[λ, [[1]]])
(Subscript[λ, [[2]]])
(Subscript[λ, [[3]]])
SelectionMove[nb, Previous, CellGroup]

Flashcard

Load the Data Sets then select the closed cell below and press 'keyboard enter'.  Default test delay period is 5 seconds and 1 trial is presented per cycle.  To change the delay period open the closed cell and change the argument in the Pause command.

nb = SelectedNotebook[];
λ = (Subscript[ω, [[Random[Integer, {1, Length[ω]}]]]]);
(Subscript[λ, [[1]]])
Pause[5]
(Subscript[λ, [[2]]])
SelectionMove[nb, Previous, CellGroup]

Reader's Index

Search & Report Module

g[t_, λ_, e_]:=Module[    
\({τ = t, z = λ, 
            y = e, Ω, Ψ}, \ \ 
Ω = 
            Table[
Length[
Flatten[
StringCases[
StringInsert[
ToString[τ\_\(\([[\)\(j, e\)\(]]\)\)], "_", \(-1\)], z ~~ _, 
                    IgnoreCase -> 
                      True]
]], 
{j, 
                Length[τ]}]; 

Ψ = 
            Table[
If[Ω\_\(\([[\)\(j\)\(]]\)\) > 
                  0, τ\_\(\([[\)\(j\)\(]]\)\)], 
{j, 
                Length[Ω]}]; 

Γ = 
            Select[
Ψ, 
Length[#] > 
                  0 &]; 

Table[{Γ\_\(\([[\)\(j, 1\)\(]]\)\), 
              PaddedForm[Γ\_\(\([[\)\(j, 2\)\(]]\)\), 
                4], Γ\_\(\([[\)\(j, 3\)\(]]\)\)}, {j, Length[Γ]}]\)
]

Off[General::"spell1"]

ri[x_]:=Module[    
\({ex = x, v, cc, s}, 
v = 
          g[κ, 
ToString[
              ex], 
3]; 
cc = 
          Complement[
            v\_\(\([[\)\(All, 1\)\(]]\)\)]; 
s[
            k_] := {β[
              k], 
TableForm[
Sort[\(Select[v, #\_\(\([[\)\(1\)\(]]\)\) == k &]\)\_\(\([[\)\(All, {2, 3}\)\(]]\)\)]]}; 
Do[
Print[
TableForm[
\(Table[s[cc\_\(\([[\)\(j\)\(]]\)\)], {j, 1, Length[cc]}]\)\_\(\([[\)\(j\)\(]]\)\)]], 
{j, 1, Length[cc]}]\)
]

Data Sets

β[x_] := 
      Switch[x, 
        1, "first author's name, first book's title, first book's \
subtitle", 
        2, "second author's name, second book's title, second book's \
subtitle"]

Copy and paste the template, below, with appropriate change to the reference number.

{2, \[Placeholder], ""},

\(\(κ = {
{2, 
            23, "fact or concept on page 23 of second book"}, 
{2, 
            12, "fact or concept on page 12 of second book"}, 
{2, 
            0, "second author's name, second book's title, second book's subtitle"}, 

{1, 
            28, "fact or concept on page 28 of first book"}, 
{1, 
            4, "fact or concept on page 4 of first book"}, 
{1, 
            0, "first author's name, first book's title, first book's subtitle"}
};\)\)

Search & Report Template

ri[
"search string"
]

Random Selection Viewer

Load the Data Sets then select the closed cell below and press 'keyboard enter'.  Default test delay period is 3 seconds, and 1 trial is presented per press.  To change the delay period, open the closed cell and change the argument in the Pause command.

nb = SelectedNotebook[];
\(\(φ = 
        Select[κ, #\_\(\([[\)\(2\)\(]]\)\) > 0 &];\)\)
λ = (Subscript[φ, [[Random[Integer, {1, Length[φ]}]]]]);
Print[β[(Subscript[λ, [[1]]])]]
\(Print["page ", λ\_\(\([[\)\(2\)\(]]\)\)]\)
(Subscript[λ, [[3]]])
SelectionMove[nb, Previous, CellGroup]

Practical Demonstrations & Questions