english

learn php

11. data storage
15 minutes / 2845 words

eleven

storing and retrieving data

somewhere in your app, it's just about guaranteed you'll have to store information somewhere and retrieve it. so far, we've only dealt with data either created inside the app or sent by the user using get or post requests.

the most common situation you'll encounter is authentication. at its simplest, that means reading and writing usernames and passwords somewhere outside your app. two other very common use cases are reading and writing articles or posts and storing or editing product information.

before we get into the specifics of how to do that, every time you design an app that involves data storage and retrieval, you have to choose where you plan to keep it. you realistically have three options. first, you can store it to files on your server. that means you come up with your own format for the data -- often something like comma-separated text or json plus binary files like photos and videos in their original formats. second, you can store it in a database like mysql or postgres -- the database deals with organization but you have to interact using a specific language the database software understands. third, you can outsource your data using an external api -- you send requests to another program, often on another server, just like we did with get and post requests.

there's no one-size-fits-all solution for data management.

the most common consideration is whether you're developing a new app or modifying an existing one. if you're not starting from scratch, there's likely a data management solution already in place and you have to make a decision about whether it's worth the effort to change to another, even if the existing approach is not the best option. php has historically been taught -- and continues to be -- as a database-model platform. there's absolutely nothing wrong with using a database but it has been pushed as the only solution for data management for so long many developers don't even consider another option. so you will almost certainly see situations where a database is being used in places it's overkill for the purpose.

in fact, a lot of php courses don't just push database storage. they don't even bother to mention other options.

the next consideration is the existing infrastructure in the organization. if, for example, everything your client uses stores its information in mysql databases, it might be a strong reason to leverage that already-built system for backups and organizing the data. if they store everything using an external data provider, that might be your best bet.

sometimes the client will demand a particular approach and give you no choice.

but, when it comes to most of your projects, you'll likely have to make a decision about where your data should live. making that decision starts with understanding the different options.

file storage

for most projects, even if this isn't the primary place to keep data, you'll keep at least some things as plain data on your server. in fact, this is the only data storage method we've already talked about. when we dealt with post requests, you uploaded and saved files on the server.

generally, it is easier to keep binary files in their original formats. that means things like photos and videos, music, zip and pdf. you can certainly store them in a database if that's useful for your needs but it's usually a better option to store them on your server.

if you're running your app across more than one server, that means you need a way to either synchronize those files or store them on a dedicated file server. but, for most smaller apps, that isn't a concern.

aside from the binary data you'll use, though, file storage is an option for your strings and numbers, too. you have a few different ways to do that. one is to use a delimited text file. if you've ever used a spreadsheet, you've probably seen comma-separated or tab-separated files. those are also extremely common as ways to store data to be used in programs.

apple,zucchini,mint
peach,broccoli,cayenne pepper
pear,cauliflower,paprika
plum,potato,cumin
cherry,turnip,cinnamon

or...

apple   zucchini    mint
peach   broccoli    cayenne pepper
pear    cauliflower paprika
plum    potato  cumin
cherry  turnip  cinnamon

give a set of data where each row is a fruit, a vegetable and a spice, first separated by commas, second by tabs.

another way to store information in text files is just to treat files like variables. this can be particularly useful when your data is long strings like blog articles or book chapters. there's no need to delimit anything if you're only reading a file directly into a variable.

a third option is to store data in an existing text format like json.

[
    {
        "fruit": "apple",
        "vegetable": "zucchini",
        "spice": "mint"
    },
    {
        "fruit": "peach",
        "vegetable": "broccoli",
        "spice": "cayenne pepper"
    },
    {
        "fruit": "pear",
        "vegetable": "cauliflower",
        "spice": "paprika"
    },
    {
        "fruit": "plum",
        "vegetable": "potato",
        "spice": "cumin"
    },
    {
        "fruit": "cherry",
        "vegetable": "turnip",
        "spice": "cinnamon"
    }
]

json usually results in a larger file but one that is easier to read and parse. php includes built-in functions for working with json that we will look at later.

the final option for storing data as files is to create your own binary format. php does, in fact, include functions like pack and unpack for dealing with binary storage. with options like json and delimited text, however, this is typically needlessly-complex. there are certainly cases where this is the best option -- google stores its search information as packed text data, for example. but they usually involve unimaginably large quantities of data and a need for extreme drive-space efficiency on a scale even most enterprise clients don't approach.

if you're seriously considering storing data in a custom binary format, what you probably need is a database.

for most projects, storing as text, usually json, is the best option. if you don't have a reason to use a different approach, i suggest making that your default answer to the question of data storage.

databases

if you're dealing with a lot of data or many different types of ways to organize it, a database might be your best option. json is great if you want to look something specific up or load a whole block of data at a time. it's not so friendly when you want to get a result like "which customers have logged in more than ten times in the last six months?" or "how many people have bought at least three items this year?". that's where a database becomes very useful.

in other words, if you're dealing with sales and reporting, you probably want to move from file storage to database storage.

the other way to picture the decision between the two is that, in practice, a json or delimited file gets treated like an array. if you picture your data inside your program as being an array, it's a great fit. if you see it as a big blob of information you need to ask specific questions to get much smaller subsets of data in response to, that's how a database works. file storage is very efficient if you need to keep dealing with all your data at once. databases work better if you're only ever looking for a tiny fraction of your data.

in the php world, we usually talk about two databases -- mysql/mariadb and postgres (or postgresql, as it's technically called, though it's rarely seen written out). mysql and mariadb are realistically the same thing in almost every way. mariadb is an open-source version of mysql. they're the default database on the web. is it because it's the best option? no. but it works and has powered a lot of the internet for decades. it's beyond battle-tested and has the widest community support. if there's a security problem, someone fixes it very quickly. postgres is a newer option and has a lot of enterprise adoption. where mysql is clean and simple, postgres focuses on data security and complex queries.

for smaller projects, you likely won't notice much difference between the two. for larger projects, you'll likely be integrating with database specialists and pre-existing database systems you have to adhere to. if you're going to use a database, it may also come down to your hosting provider. if you're sharing a server with other projects, it likely doesn't run both mysql/mariadb and postgres. so your decision will be made for you.

either way, they both communicate using the sql database language so your task doesn't change very much. there are significant technical differences but they're far beyond the scope of learning to code in php.

it might be useful to know there are other database options like oracle and microsoft sql server out there. if you're developing in php, i suspect you won't encounter them very often if at all. if you end up coding in a windows environment, microsoft sql server is common but you probably won't be writing your software in php at that point.

apis

external data storage solutions typically communicate using apis -- application programming interfaces. that's just a technical name for a program that interfaces with the outside world. much like you received get and post requests and processed the data inside, apis allow you to send requests and receive data in return. in much the same way as "save this to a file" or "read a file from the drive", you can send and receive data from inside your app to a dedicated storage provider's api.

if you're writing an ecommerce app, this is guaranteed to be a part of it because you don't do the processing of the credit cards on your server. you have to, at some point, send that information to a credit card processor. that might be a bank or a dedicated service like paypal or stripe. regardless, you're sending data like "here's a person who's paying this amount of money" and getting back "their transaction was successfully processed".

external data storage can be used for other types of information, though. many organizations use amazon s3 or microsoft azure for their data instead of keeping it on their own servers. thankfully, sending data to other servers is relatively simple using php's built-in functions and most data providers have their own interfaces that can be included in your app to make integration easier.

files and permissions

if you're going to work with files on a web server, understanding permissions is an absolute necessity. your server administrator might handle it for you but it's probably something you'll have to at least be able to talk about intelligently in relation to your app. plus, it's not complicated.

given that we're talking about a web server for php, it's a safe bet it's going to be some form of linux. all of this will apply the same to any distribution of linux or bsd as well as any unix-type operating system like macos.

the linux file structure is a series of directories with files inside them. the base directory of the entire filesystem is / and every directory, file, piece of hardware, network connection and external server is treated the same way -- as a string of text beginning with /. (there are some exceptions for network requests where they specify the protocol before the slash but the concept remains the same and those are not relevant to this topic.)

for example, your home folder might be /home/username, your first hard drive might be /dev/hda and your main network interface might be /dev/eth0. unlike on a windows system, which many people are already used to, everything on a linux, mac or unix-derived system functions as a part of the filesystem structure.

that makes the question of permissions a lot easier to get your head around.

everything has three categories of permissions. one for the owner, one for the owner's group and another for everyone else with access to the system. in the case of a website, the owner is often a user specifically created for the site, the group a group including the users attached to the websites for that server and the rest... well, i think we understand what everyone else means already. it's also not unusual for the group to only include a single user, depending on how the security profiles on the server are set up.

of course, this same pattern applies to your local computer if that's where you're running your php. but the same principles apply as long as it's unix-derived. you can certainly run php on a windows desktop if that's what you want to do but it often behaves slightly differently and you definitely won't be running it on a windows server so it's a good idea to either start using linux on your pc or connect to a linux server. even an old computer on your home network is totally fine or linux in a virtual machine. this doesn't have to cost you money.

each of those three categories -- user, group, everyone -- is assigned three abilities -- read, write, execute. read is the ability to see the contents of a file or directory. write is the ability to change the contents of a file or create a new file if we're talking about a directory. execute is the only one that's a little misleading in its name when it comes to the web. it's the ability to run a program on the machine. but php doesn't run the files as programs. it just loads them the same way a word processor opens a document. so execute doesn't matter for php in practice.

sometimes you'll see this written as nine letters -- rwx-rwx-rwx -- with some of the letters removed and replaced with -. if you see something listed as rw--r---r--, that means the owner can read and write, the group can read and everyone can read.

before we get too wrapped up in "everyone", this means everyone who has access to the server as a user. it doesn't mean the general public who can't log in. so what may seem very insecure (everyone being able to read a file), in practice, likely doesn't mean much.

once you understand the concept, it's time to introduce how these permissions are usually written. the nine-letter model is easy to read but unwieldy to type so it's usually shorted to three numbers with each digit being between 0 and 7. read counts as 4, write as 2 and execute as 1. no access is 0. add them up and you can get any combination you like.

that means 740 gives the user read, write and execute, the group read and everyone else no access at all. the two typical permissions settings you'll see in the world of php are 644 and 600. sometimes you'll see 744 and 700 and, for all intents and purposes, these mean the same thing as far as the php content goes.

why is this important to understand at this point in your php journey? if you are going to read and write files, you have to have access to do those activities. if you try to write to a directory or file you don't have access to, it doesn't matter if your php code is perfect. you'll still get an error or, sometimes, just no result. if you try to read, you might get an error, quite often, an empty return as if the file is blank. in other words, check your permissions before you do anything with files and you'll thank yourself for the saved headaches.

filesystem locations

where your php files are stored can vary wildly not just with the distribution of linux or macos you're running but with the web server's software. it could be something as simple as /www/httpdocs/ or as complex as /var/www/vhosts/php.org/httproot/. thankfully, though, you're putting your files in a predetermined location where the software is looking for them and it won't change on you without you intentionally changing something.

how you talk about that location in php, however, isn't nearly as fluid or unpredictable.

we've already seen how to access the directory the script is running in -- __DIR__ always returns the script's directory. note that it doesn't include a trailing /. if you want to store a file named text.txt in the directory of the script, you have to save it as __DIR__ . "/text.txt".

relative positions are given using .. and /. to go one directory up then into a directory called uploads, you could save the file as __DIR__ . "/../uploads/text.txt". if your script is in /www/httpdocs/php, that will write a text file called /www/uploads/text.txt. this might give you a hint about the danger inherent in working with the filesystem. if you ever let a user give you text to use anywhere in that string, they could include a bunch of ../../.. that would get them to places on the server they shouldn't be able to see or change. this is a very real security concern and it's important to mitigate it.

assignments

© avi sato. creative commons attribution-noncommercial-noderivatives.