[EE]: How do you organize your 'notes'
Locked
[EE]: How do you organize your 'notes'
 
|
The 'hiring firmware people' thread got me to thinking about asking this
question: How do you organize your notes so you can find them later for reference? At the moment I have about 7 years worth of notes with code and links and other references kept inside Emacs and org-mode (it's text base so I can look at it with any editor). I use a lot of features such as links (URLs and local file links and same file links), block mode (usually executable code fragments) and notebooks pointing to notebooks pointing to notebooks. Problem is that I'm beginning to find that I can't recall some detail that would lead me back to the specific notes. Lots of my projects have odd names or worse geek-code (letters and numbers - not as bad as UUIDs). I have attempted to use MS notes but find it annoying and limited (no lisp ;-) ). So I tend to take my org notes and print them out to a nice format MS Notes can use and stored them there. But Note's search is worse than my gigs of org files. I do have Unix tools on Windows and I have been using grep -rn . -e 'whatever' which helps but as the data grows the time it takes is getting worse. -- Linux Home Automation Neil Cherry [hidden email] http://www.linuxha.com/ Main site http://linuxha.blogspot.com/ My HA Blog Author of: Linux Smart Homes For Dummies -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
I've been using emacs in text mode for a few decades. A single file
for work and play. New stuff is put at the top. I've macros for inserting the date and time. I link out to the filesystem for code greater than a line or two. My current file was started 2009-11-03 when I joined OLPC, and is 112597 lines as of today. That's about 30 lines a day. Whenever I've struggled to find something, when I do find it I add the keywords that I had expected to find it with. This tunes my search practice to compensate for normal memory loss and reassociations. Failed searches are inserted at top of the file; they are an activity after all. The file is not under change control, but there are multiple backups. I'd call it a journal, or an engineering notebook. Some projects have their own file, which ceases to be maintained once the project becomes idle. Usually no more than a few hundred lines each, and about 132 of them on my system today. I've keyboard macros that open selected string as a URL, or as a file in a new buffer. -- James Cameron http://quozl.netrek.org/ -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
In reply to this post by Neil Cherry-3
I saw Russell's when I visited him, and in the interest of improving his
security, I'll share it here (you're welcome, Rus). He has tons of nested folders where he downloads everything from PDFs to web pages with important research and images of circuit diagrams. They are all filed in a hierarchy under project names, research topic, or year, whichever is most relevant to the collection. He then uses shell commands to quickly browse through the folders of files and something like irfanview to popup images to show people. Years of practice has made him quite efficient at this, and his monitors are "bloody huge" in physical size which helps with all the text. Additionally, he supplements this with nested bookmark folders for active research and links he wishes to keep live (Rus, feel free to clarify this distinction). Am I close? As for me, I just remember everything by keywords that I later type back into google (yes, for the last 10 or so years of information, our brains are awesome). It's surprisingly worked for me, except for in private intranets at work where I have to use browser bookmarks. On Wed, 18 Nov 2020 at 12:10, Neil Cherry <[hidden email]> wrote: > The 'hiring firmware people' thread got me to thinking about asking this > question: > > How do you organize your notes so you can find them later for reference? > > At the moment I have about 7 years worth of notes with code and links and > other references kept inside Emacs and org-mode (it's text base so I can > look at it with any editor). I use a lot of features such as links (URLs > and local file links and same file links), block mode (usually executable > code fragments) and notebooks pointing to notebooks pointing to notebooks. > Problem is that I'm beginning to find that I can't recall some detail that > would lead me back to the specific notes. Lots of my projects have odd > names or worse geek-code (letters and numbers - not as bad as UUIDs). > > I have attempted to use MS notes but find it annoying and limited (no > lisp ;-) ). So I tend to take my org notes and print them out to a nice > format MS Notes can use and stored them there. But Note's search is worse > than my gigs of org files. I do have Unix tools on Windows and I have been > using grep -rn . -e 'whatever' which helps but as the data grows the time > it takes is getting worse. > > -- > Linux Home Automation Neil Cherry [hidden email] > http://www.linuxha.com/ Main site > http://linuxha.blogspot.com/ My HA Blog > Author of: Linux Smart Homes For Dummies > -- > http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive > View/change your membership options at > http://mailman.mit.edu/mailman/listinfo/piclist > http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
In reply to this post by James Cameron-2
On 11/17/20 6:47 PM, James Cameron wrote:
> I've been using emacs in text mode for a few decades. A single file > for work and play. New stuff is put at the top. I've macros for > inserting the date and time. I link out to the filesystem for code > greater than a line or two. > > My current file was started 2009-11-03 when I joined OLPC, and is > 112597 lines as of today. That's about 30 lines a day. > > Whenever I've struggled to find something, when I do find it I add the > keywords that I had expected to find it with. This tunes my search > practice to compensate for normal memory loss and reassociations. > Failed searches are inserted at top of the file; they are an activity > after all. > > The file is not under change control, but there are multiple backups. > > I'd call it a journal, or an engineering notebook. > > Some projects have their own file, which ceases to be maintained once > the project becomes idle. Usually no more than a few hundred lines > each, and about 132 of them on my system today. > > I've keyboard macros that open selected string as a URL, or as a file > in a new buffer. > Thanks Look into org-mode, it has all the above and quite a bit more. But be careful it's a rabbit hole. I can't quite ignore the projects, they come back again and again. They're the thing I most need to find. I have projects dating back to 2016 at this point and they're consuming 88M of text. My home machine is a bit better in that I have the notebooks a bit more organized. I can also convert them to HTML and post them to my web pages. I can use google to search those (no proprietary information there). I can't do that with my work notes. Right now the notes system works well but I am failing at organizing them. I may need to do better with keywords though a lot of my work covers multiple keywords. I also have all my notebooks linked at a main notebook so I can find them. But it is the specific information inside that's the difficult part to find now. Mine isn't under change control yet either (I won't go into why, grumble grumble grumble). -- Linux Home Automation Neil Cherry [hidden email] http://www.linuxha.com/ Main site http://linuxha.blogspot.com/ My HA Blog Author of: Linux Smart Homes For Dummies -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
|
|
Hi Neil. Long time lurker etc...
On Wed, Nov 18, 2020 at 9:07 AM Neil Cherry <[hidden email]> wrote: > Look into org-mode, it has all the above and quite a bit more. But be > careful > it's a rabbit hole. > . . . > Right now the notes system works well but I am failing at organizing them. > I may > need to do better with keywords though a lot of my work covers multiple > keywords. > I also have all my notebooks linked at a main notebook so I can find them. > But it > is the specific information inside that's the difficult part to find now. > > I have customer service requests in 10,039 HTML files on my employer's Omega. It's a search engine based on Xapian (xapian.org), running on Ubuntu Linux, and it does a good job of locating stuff in that huge pile. I have a Python script that checks for changes every three hours and creates new HTML pages as needed. You could probably set up something similar by exporting your org files to html. I use org-mode for my day-to-day journal; I should take my own advice and put those files into Omega. /Joe -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
In reply to this post by Ryan O'Connor
I use ZIM.EXE.
Open Source, free. I think it is used by Wikipedia. I am 80 years old. My short term memory is very elusive. I build or collect "cheat Sheets" when I try to write code. Messy, but it seems to work for me. The stack of 3x5 note cards scattered around my house to remind me of open projects. I am easily distracted. If you are fortunate enough you may grow old with toys and time . I think it was Will Rogers who offered "You can deal with things as they are or you can deal with them as they are not"' On 11/17/2020 7:17 PM, Ryan O'Connor wrote: > How do you organize your notes so you can find them later for reference? -- John Ferrell W8CCW Julian NC 27283 It is better to walk alone, than with a crowd going the wrong direction. --Diane Grant -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
On 11/25/20 11:13 PM, John Ferrell wrote:
> I use ZIM.EXE. > > Open Source, free. I think it is used by Wikipedia. > > I am 80 years old. My short term memory is very elusive. I build or > collect "cheat Sheets" when I try to write code. Messy, but it seems to > work for me. The stack of 3x5 note cards scattered around my house to > remind me of open projects. I am easily distracted. I'm not distracted I'm working on too many projects and ... wait, what was that ? ;-) > If you are fortunate enough you may grow old with toys and time . I > think it was Will Rogers who offered "You can deal with things as they > are or you can deal with them as they are not"' I'm getting there. ;-) Sorry John, I'm using Emacs. I wasn't asking about the application but rather the organization methods. Org-mode, Emacs and lisp provide an extremely flexible way to present data yet store it as plain text or to publish to more complex formats such as pdf or HTML. The problem is how to organize the data so it's easy to find several months later after I've been in several other projects. Many times it's code which is difficult to search for as I need to remember some exact detail to find that segment of code. It has been suggest that I use tags. At the moment I don't know how to use them or organize them yet. Lately I've gone to putting notes in two places with a file link in my main notebook. The links (just like URLs) make it easy to find the file. The tags should make it easier to find the material in the files. By adding it in two places it does make it a bit easier to recall what I was looking for. If I can figure out how to 'publish' the material to the company wiki in an automated fashion I'll really have an excellent solution. I'll then need to work out the layers of security for the wiki pages. -- Linux Home Automation Neil Cherry [hidden email] http://www.linuxha.com/ Main site http://linuxha.blogspot.com/ My HA Blog Author of: Linux Smart Homes For Dummies -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
On Thu, Nov 26, 2020 at 03:07:16AM -0500, Neil Cherry wrote:
> [...] It has been suggest that I use tags. > At the moment I don't know how to use them or organize them yet. > [...] > If I can figure out how to 'publish' the material to the company > wiki in an automated fashion I'll really have an excellent solution. Some of my files use markdown format, and so I can convert them automatically to PDF, HTML, or some Wiki markups. I use the Debian "discount" package to convert markdown to HTML, and "wkhtmltopdf" to convert to PDF. For Wiki output, I've used "pandoc". -- James Cameron http://quozl.netrek.org/ -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
On 11/26/20 3:52 AM, James Cameron wrote:
> On Thu, Nov 26, 2020 at 03:07:16AM -0500, Neil Cherry wrote: >> [...] It has been suggest that I use tags. >> At the moment I don't know how to use them or organize them yet. >> [...] >> If I can figure out how to 'publish' the material to the company >> wiki in an automated fashion I'll really have an excellent solution. > > Some of my files use markdown format, and so I can convert them > automatically to PDF, HTML, or some Wiki markups. > > I use the Debian "discount" package to convert markdown to HTML, and > "wkhtmltopdf" to convert to PDF. For Wiki output, I've used "pandoc". > Emacs/org-mode uses Pandoc also to convert the files. The company wiki has their own mark up but also supports HTML (with css and javascript). They just don't have an API to publish a document from other applications. I can attach but not exactly import. I may use Robot Framework to put the file into the wiki after it have been converted with Pandoc. -- Linux Home Automation Neil Cherry [hidden email] http://www.linuxha.com/ Main site http://linuxha.blogspot.com/ My HA Blog Author of: Linux Smart Homes For Dummies -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
In reply to this post by Neil Cherry-3
Emacs and org-mode have been mentioned by others and I also use both. My
habit has been to keep a yearly org file that I record daily notes, but I also keep individual org files for notes that are searched through using the Deft package: https://jblevins.org/projects/deft/ In the course of working, there are notes I refer to frequently and Deft makes it easy to find them, all from within Emacs. Take care. On Tue, Nov 17, 2020 at 6:10 PM Neil Cherry <[hidden email]> wrote: > The 'hiring firmware people' thread got me to thinking about asking this > question: > > How do you organize your notes so you can find them later for reference? > > At the moment I have about 7 years worth of notes with code and links and > other references kept inside Emacs and org-mode (it's text base so I can > look at it with any editor). I use a lot of features such as links (URLs > and local file links and same file links), block mode (usually executable > code fragments) and notebooks pointing to notebooks pointing to notebooks. > Problem is that I'm beginning to find that I can't recall some detail that > would lead me back to the specific notes. Lots of my projects have odd > names or worse geek-code (letters and numbers - not as bad as UUIDs). > > I have attempted to use MS notes but find it annoying and limited (no > lisp ;-) ). So I tend to take my org notes and print them out to a nice > format MS Notes can use and stored them there. But Note's search is worse > than my gigs of org files. I do have Unix tools on Windows and I have been > using grep -rn . -e 'whatever' which helps but as the data grows the time > it takes is getting worse. > > -- > Linux Home Automation Neil Cherry [hidden email] > http://www.linuxha.com/ Main site > http://linuxha.blogspot.com/ My HA Blog > Author of: Linux Smart Homes For Dummies > -- > http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive > View/change your membership options at > http://mailman.mit.edu/mailman/listinfo/piclist > http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
On 11/28/20 10:09 AM, Matthew Miller wrote:
> Emacs and org-mode have been mentioned by others and I also use both. My Love Emacs and org-mode. :-) > habit has been to keep a yearly org file that I record daily notes, but I > also keep individual org files for notes that are searched through using > the Deft package: > https://jblevins.org/projects/deft/ > In the course of working, there are notes I refer to frequently and Deft > makes it easy to find them, all from within Emacs. Thanks I'll take a serious look at deft. -- Linux Home Automation Neil Cherry [hidden email] http://www.linuxha.com/ Main site http://linuxha.blogspot.com/ My HA Blog Author of: Linux Smart Homes For Dummies -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
|
|
In reply to this post by Ryan O'Connor
This is a topic very near and dear to me, so pardon my enthusiasm....
After bouncing around so many systems from an Apple IIe to the first PC (loved Sidekick) to the latest Linux, I have a strong requirement on not depending on a particular 'package' to operate. Process is more important. * First, my notes have high value. They should not be held in some proprietary format. * The notes may have different context or meaning, or my frame of reference may be different, years from now ( I may think 555 means timer chip today, but 10 years from now I may have forgotten that and it's now just 'timer' or a 69sxxx device (whatever that will be...). This also means structure and hierarchy is less important. * I should have a primary interface, and an equally functional alternative way to access it (in case my chosen one becomes an abandoned software package). * It shouldn't take a lot of my time, or I end up pre-filtering the value of notes and some good ones never get captured. That means probably hotkeys and clipboard usage. * As a bonus, if it can capture alternative MIME types, great. * Any solution should have graceful degradation in case one or another dependencies is unavailable. So, my solution has been to center around 2 tools that have cross-platform availability, are for convenience, and a process that doesn't absolutely depend on them. The 'nice' tools are zim (cross platform) and copyq (using Linux here, may also be cross-platform). [stay with me here...] I set up hotkeys and a folder. I also have project and topic folders at a very high level, but not necessary. A relatively unknown feature of zim is that it will not only save its notes as text files, but will integrate any text files in its folder into its UI and index them. It also has a built in webserver, and a useful search function. Copyq is able to save whatever MIME types it copies, keeps a queue of them as long as you specify, and each entry is addressable. The hotkeys call a script which just saves whatever data just got copied, such as: copyq read text/html 0 >$popup.$timestamp.html and copyq read text/text 0 >$popup.$timestamp.txt I am prompted for a filename at the hotkey which is $popup. copyq read image/png [or image/jpeg] 0 - saves images Very handy. Looping through locations from 0 (last clip) to n with each MIME type saves a good history should one want it. FYI Having it save into the zim folder of choice in the hotkey script automatically puts it in zim where it is picked up, pretty and searchable and with all others (and easily edited). The filename is somewhat descriptive, and the date-timestamp assures no overwrites and an indication of age. So, select - hotkey - enter a description - done. It's now in the 'database'. I also have it in html if I need that, but rarely do in notes. I will explain html ones in a minute. If one day zim is dead, grep works as it's all plaintext. If grep is gone or not on the system (let's say I put it all on my android phone), then the filename gives a clue as well as datestamp. If my names are not helpful, I even have 'about when I saved it' to go on... Graceful degradation for both environment and user degradation... ;) Really like web access to it? Simple one-line, no-setup webservers like webfsd give you the same indexing of the folders. No copyq available? 'xsel -b' will do with a little less outside function. And there are other clipboard utilities that are similar to copyq (clipit for example). One day the perfect program comes along and you want to port it? It's all plain text. Go at it... I suggest keeping the basic processes here and having whatever new fancy tool pull from it to maintain portability. Not on a device that supports a GUI (like maybe a basic ssh session), use grep. ssh back into your system from your phone and get your notes as well... For easy GUI use, it's all there in zim too, as well as by browser lookup with its built in server. Now, that brings me to the other notes - html (mostly form the web). I have 2 methods with the same philosophy. The first depends on copyq a bit more because it captures all MIME types from the source it copies, so you get html source as well as text. I copy away at all I want on a topic (doing research), and a script and a hotkey do a loop on the number I enter and essentially write a generic html page header, cat the locations I copied (say, 0 to 6 for 7 html clips), adds a generic /html to the end and I have a single page of all notes on a subject. The nice part is that I selected the parts from the pages and ctrl-c is all I needed to do - for each thing I wanted, eliminating the trash. Then hotkey and it cats it all into a file with name I provide (with automatic datecode again). It renders beautifully 95+% of the time, and captures the content I wanted 100% of the time. If I want to turn it into a pdf, just print from browser. Single docs on the topic of choice. Want something in the zim folder for that as well, and just the text notes? Links -dump or lynx -dump output directed to a file works, as does copyq text/text >filename. Lastly, I've found great use of Singlefile, which is available both as Chrome/Chromium or Firefox extension as well as scriptable code: https://github.com/gildas-lormeau/SingleFile . If a page is really good and deep, one click and I have a file that contains the images embedded, et al, without the usual folder of images and media 'file-save as' produces. One page-one file, with title as filename automatically. So far, notes from over a decade are findable in seconds on many topics, and vanishing web content is also archived, all with just a click or hotkey. Advice: invest in more storage... ;) Find and grep allow anything to be found, and in fact I have another script that uses either/both to search the html notes and open a browser with each page it finds in a tab. I can then review all for what I want. Think about that - one command or hotkey, enter a search term, get a browser with all of your saved html pages open in tabs... Like just browsing? A simple webserver like webfsd will present your html collection to wherever your browser is if not local (you wouldn't even need that locally). Image handling is probably the weakest in all this, but tools like gThumb and such seem to fill in enough. The whole idea is a balance between speed in archiving things and ease of getting it back out under any condition, including degraded software or UI or user. The last benefit is in future-proofing your notes. Should you want to splinter or re-organize your notes, it's very easy and can be automated, including multiple copies under different project folders if desired. Tools like lsync can automate processing based on content - a write to the notes folders triggers some processing and sorting or such. Being a simple, common back-end let's you do anything at any time. With multiple options for the front-end, the notes are accessible from almost anywhere on any device. And single commandlines can do things like turn every note on a topic in your notes into a pdf if you want that. This is a very functional system (I can make over a hundred notes in a day) and just touches on the automation it can support (like linking the html saves back into zim for instance). Of course these methods also allow you to easily integrate web scraping into any 'notes' you want to keep - also handy - and I leave that and more to your imagination... I hope it helps. -Dr Skip On Wed, 18 Nov 2020 at 12:10, Neil Cherry <[hidden email]> wrote: How do you organize your notes so you can find them later for reference? -- http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
Re: [EE]: How do you organize your 'notes'
|
|
In reply to this post by Neil Cherry-3
Neil,
I can't recommend Deft enough. I have a capture template setup so that C-c c and then k opens up a new org file under the Deft directory. This makes it quick and easy to add a new note. The deft part of the capture set is: (setq org-capture-templates (quote (("t" "todo" entry (file "~/org/refile.org") ;; ..... ("k" "KB FRAG" entry (function (lambda () (let ((new-file (concat "~/deft-notes/" (format-time-string "%Y-%m-%dT%H-%M-%S") ".org"))) (set-buffer (org-capture-target-buffer new-file)) ) ) )) ))) HTH On Sat, Nov 28, 2020 at 10:11 PM Neil Cherry <[hidden email]> wrote: > On 11/28/20 10:09 AM, Matthew Miller wrote: > > Emacs and org-mode have been mentioned by others and I also use both. My > > Love Emacs and org-mode. :-) > > > habit has been to keep a yearly org file that I record daily notes, but I > > also keep individual org files for notes that are searched through using > > the Deft package: > > https://jblevins.org/projects/deft/ > > In the course of working, there are notes I refer to frequently and Deft > > makes it easy to find them, all from within Emacs. > > Thanks I'll take a serious look at deft. > > -- > Linux Home Automation Neil Cherry [hidden email] > http://www.linuxha.com/ Main site > http://linuxha.blogspot.com/ My HA Blog > Author of: Linux Smart Homes For Dummies > http://www.piclist.com/techref/piclist PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist |
«
Return to MicroControllers - PIC
|
1 view|%1 views
| Free forum by Nabble | Edit this page |