Building a component management system.
- Current system
So I have accumulated a bit of a collection of electronic components, developer boards and all sorts of electronics parts for various projets and managing these has been a mess so far. Most of them are still in these boxes that they were delivered in, lying around in my place. I have managed to label a small portion of them with a part number and added them to an excel sheet, but I’m not really satisfied with this solution.
I also tried a bunch of free component management systems and everytime they seemed to be missing some key functionality that I need/want. There was a one potential open source component management system called Partkeepr that seems to have everything.
- I tried setting that up first to my Windows machine, got a bunch of errors and on the website it says: Windows is not supported, so proceed on your own risk. so I gave up on that idea.
- Then I decided to try to install it on a VPS ( virtual private server ). Spinned up my droplet on DigitalOcean and tried to setup Partkeepr there. Turns out that the composer tool that the setup uses, requires something like 4GB of RAM to work. That would be pretty costly to run on a VPS.
- Lastly I setup my raspberry pi, hoping that I could run Partkeepr on that. I spent the whole night trying to get the partkeepr to work. I wrote my notes here, although they got left unfinished:
127 lines of notes.
If I remember correctly at the end I had an issue where the install system would complain about some parameters.php file not being on the file system, and to generate that said file, I must visit the install system that doesn’t work without that file..
Then I noticed that on the PartKeepr website there are instructions on how to run PartKeepr from Git. The instructions were pretty short so I decided to try that. Again, errors after errors so I just gave up. Maybe I was doing something very wrong here or it just really is a overly complex system. The project owners also seem to have stopped maintaining the project 3 years ago.
Old component management system (.csv file)
Components labeled with a part number
I decided on the part numbering system years ago without giving it much of thought and I’m too lazy to change it at this point. Basically all the dev boards, shields and so on has a ‘A’ prefix, electronic components has a ‘B’ prefix, misc parts has a ‘C’ prefix. The numbers just increment from 1.
In the component storage system I’ll probably add zero padding to the front of the part number to make them all equal lengths. So the part numbers will be something like B0000137, B0000007 and so on. The label it self doesn’t need the zero padding.
Storage organizers for the parts
React front end up
Copy pasted some basic React setup from one of my old projects that was left unfinished. Basic routing for the navigation bar was done already and some styles + themes to make the site not look like a blank white page, just had to scrap some stuff away to make it a good starting point for this project.
Got a database setup for the project. I got the backend code from my old project as well, which also used mysql for a database so after a few struggles I got a data flowing from the front end to the database.
I just noticed that also the database seems to still include a table called ‘blog_posts’ from the old project, even though I just installed the MySQL server 😁 Not sure if old backend code managed to do a backend call or are the databases saved to the same place and the old tables were loaded.
I flashed a new Linux image to a SD card and booted it with one of the raspberry pi’s that I have lying around. I got the React + NodeJS + MySQL setup to run on there as well but building the code is so slow so I moved my dev environment to my Windows machine.
Couldn’t really decide on what to work on first, so I decided to build a dynamic table to show all the components. There is no pagination yet though, I’ll probably do that later when I start to have lots of actual components in the database. I’m using React bootstrap front-end framework for the project and it seemed to have a ready made pagination component so I’ll probably use that one.
I decided to hide the ‘add component’ functionality to a modal. I don’t really like to use modals but for this I think it’s better than to redirect to another page for example. Also some sort of drop downs for long forms are annoying.
One functionality that wasn’t done very well or was missing completely from the free alternatives, is the ability to attach files to a component. For example images, datasheets, example code/drives and various zip files.
Testig drag and drop file upload
Preview images when dropped
Sending the files to the backend for storing
Using requests from superagent package to handle the file upload. File storage is a folder on the repository for now ( added in .gitignore).
Styling the drag and drop box
Upload folder structure
It was a bit more complex to make this work than I originally thought. Some key notes:
- Basically when a component is added, the input field data, and all the files are sent as a separate requests to the backend.
- I couldn’t figure out how to add the part number or some ID to the file post request.
- The file upload requests sometimes ended up to the backend first and since they are without an ID they need to wait for the component data request to arrive.
The file upload still needs some work, but it’s good enough at least for now.
Component view + storage folder
Attachment file can be downloaded by clicking the filename.
I have been working on this project for a week now, a couple of hours in the evenings and it seems to be moving forward quite nicely, learned a lot so far. I’ll probably end this post here and do a some ‘component storage update’ post when I have more progress done and pictures taken.
I don’t like to keep writing these posts as I go along with the projects so I just write these in one sitting and be done with them. I just decided that this is a good break point to write things down before I forget them.
The component storage system obviously has alot of things left to do and a few key functionalities missing like search and an edit component functions, more on that in the TODO section.
Not a complete list, just throwing things here from the top of my head.
- Setup dev env
- Front-end and backend setup
- Preliminary component listing page
- Preliminary component view page
- Preliminary component add function
- Preliminary file uploader
- File handling on the backend
- Search component-function
- Edit component-function
- Figure out proper names for component data fields
- Add proper layout for the single component view page.
- File upload system in ‘add component’ view needs an option to select thumbnail picture
- Item type selector to the ‘add component’-view ( component, dev board, misc ).
- Change input fields based on the type in the ‘add component’-view.
- Add tests
- Remove files-button in add component-view.
- Re-structure backend code
- Bulk import components from .csv
- Image / pdf / text file viewer to the single component view page
- Auto backup system for the database and attachment files