简介
数据库存储选型经验总结 未读
数据库治理的云原生之道 —— Database Mesh 2.0
数据的组织
假定数据库存放数据的文件称为data file，数据库的内容在内存里是有缓存的，称为db buffer。某次操作，我们取了数据库某表格中的数据，这个数据会在内存中缓存一些时间。对这个数据的修改在开始时候也只是修改在内存中的内容。当db buffer已满或者遇到其他的情况，这些数据会写入data file。(为了维护这个db buffer和db file的一致性，引入了checkpoint)
db buffer的存在不仅提高了性能，系统也有机会对db buffer进行一定整理后集中写入，毕竟db数据随机写入的开销比较大。log file一般是追加内容，可以认为是顺序写，
How do you build a database
摘自How do you build a database? ，为防止链接失效，贴上原文。
Its a great question, and deserves a long answer. Most database servers are built in C, and store data using B-tree type constructs. In the old days there was a product called C-Isam (c library for an indexed sequential access method) which is a low level library to help C programmers write data in B-tree format. So you need to know about btrees and understand what these are. BTree 很重要
Most databases store data separate to indexes. Lets assume a record (or row) is 800 bytes long and you write 5 rows of data to a file. If the row contains columns such as first name, last name, address etc. and you want to search for a specific record by last name, you can open the file and sequentially search through each record but this is very slow. Instead you open an index file which just contains the lastname and the position of the record in the data file. Then when you have the position you open the data file, lseek to that position and read the data. Because index data is very small it is much quicker to search through index files. Also as the index files are stored in btrees in it very quick to effectively do a quicksearch (divide and conquer) to find the record you are looking for. 一个表单单数据文件是不够的，需要一/多个索引文件。
So you understand for one “table” you will have a data file with the data and one (or many) index files. The first index file could be for lastname, the next could be to search by SS number etc. When the user defines their query to get some data, they decide which index file to search through. If you can find any info on C-ISAM (there used to be an open source version (or cheap commercial) called D-ISAM) you will understand this concept quite well.
Once you have stored data and have index files, using an ISAM type approach allows you to GET a record based on a value, or PUT a new record. However modern database servers all support SQL, so you need an SQL parser that translates the SQL statement into a sequence of related GETs. SQL may join 2 tables so an optimizer（优化器最初是为了加快join表的速度么？） is also needed to decide which table to read first (normally based on number of rows in each table and indexes available) and how to relate it to the next table. SQL can INSERT data so you need to parse that into PUT statements but it can also combine multiple INSERTS into transactions so you need a transaction manager to control this, and you will need transaction logs to store wip/completed transactions.
It is possible you will need some backup/restore commands to backup your data files and index files and maybe also your transaction log files, and if you really want to go for it you could write some replication tools to read your transaction log and replicate the transactions to a backup database on a different server. Note if you want your client programs (for example an SQL UI like phpmyadmin) to reside on separate machine than your database server you will need to write a connection manager that sends the SQL requests over TCP/IP to your server, then authenticate it using some credentials, parse the request, run your GETS and send back the data to the client. So these database servers can be a lot of work, especially for one person. But you can create simple versions of these tools one at a time. Start with how to store data and indexes, and how to retrieve data using an ISAM type interface. There are books out there - look for older books on mysql and msql, look for anything on google re btrees and isam, look for open source C libraries that already do isam. Get a good understanding on file IO on a linux machine using C. Many commercial databases now dont even use the filesystem for their data files because of cacheing issues - they write directly to raw disk. You want to just write to files initially. I hope this helps a little bit.
概要内容：
- 知道BTree 很重要
- 一个表单单数据文件是不够的，需要一/多个索引文件，数据文件和索引文件分开存储
- 有了数据文件和索引文件，你就可以读写数据了，但你需要SQL parser 将sql 翻译成读写操作，需要optimizer加快join表的速度，需要Transaction manager 管理事务
- 备份儿/恢复数据文件、索引文件、Transaction log文件。如果支持客户端程序的话，还需要一个Connection manager