there are two schools of thought about dealing with storing data as plain text files. one is to have many small files to keep them easy to navigate and efficient to parse. the other is to have a small number of very large files but only read small pieces of them at a time.
the second approach certainly works in some situations but none of those situations apply to using a runtime-compiled language like php. it can absolutely be done with the built-in commands for php and many people do it. but it is consistently a bad idea and leads to unnecessary complexity in the name of trying to avoid having a lot of small files to read and write. if you think the quantity of files is truly going to get that large (think many billions or higher), plain text files directly accessed by php on the filesystem is the wrong solution and you should either be looking at a database or an external storage provider. partial file reading of text files in php is a solution looking for a problem that simply doesn't exist in the real world.
we will only be using the first approach here. that doesn't mean you're confined to small amounts of data or small projects. just that, if your quantity of data is going to be truly massive, text files on the web server is not the answer. php has better solutions for those tasks than that and we will go through those shortly.
before you get into the mechanics of reading and writing the files, it's important to think about how you're going to organize them on the server and what you're going to use as a naming scheme.
let's take a look at two possible examples -- a blog and a photo gallery.
the first thought might be that you want to have one text file for each blog entry and use the name of the post as the filename. you then have to ask yourself an important question. if the files are going to be named with the names of the posts, that means the names have to be unique -- is that going to be a problem? what happens when someone wants to write a post called "happy new year!" or "back to school"? maybe you do want the post names to be unique. but it's something to consider.
maybe you want the files to be named by date and time. if you name them with the date and time in hours and minutes, that means a maximum of one post a minute. include seconds and now you can have one post per second, etc. none of these are problems, just potential limitations you have to be aware of when you're thinking of your naming scheme.
maybe you want to combine the two approaches. have each file look like 2073.07.20.14.23.50 summer thoughts.txt. that way, you're limited to one identical post title per second. that also gives you the ability to see the date and time as well as the name of the post without having to read the contents of the file, definitely a benefit in terms of efficiency.
another alternative is to name the files with unique identifiers so there's never a possibility of a duplicate. php has some built-in functions for generating unique strings of text. the simplest one is uniqid, which returns a semi-unique string. as long as you're not generating more than one per microsecond, it's unique enough for naming files. it can generate either 13 or 23 characters of semi-random unique string.
uniqid(); // 13 characters
uniqid("", true); // 23 charactersthe first parameter is an optional prefix. the second is whether to generate an extra ten characters. both default to empty/false.
$filename = __DIR__ . "/posts/" . uniqid("", true) . ".txt";depending on your web root, this might generate something like /www/httpdocs/posts/6a8d04da310c29.86234472.txt.
if your blog is multiuser, you might need to think about putting each user's posts in a separate subdirectory. if you're posting once a day, keeping the files in a single directory per user might be sustainable for decades. if you're microblogging a few times a minute, that number of files will add up very fast and get overwhelming to organize so you might want to separate the files by year, month and even day.
your specific use-case will determine the best filename pattern.
much different from the blog situation, now you have an issue of more than one file per post. sure, you might be adding images or videos to blog posts but they're not necessarily attached to single posts. they can be uploaded as files to be referenced. for your photo gallery, though, you need to be able to attach descriptive content to the uploaded photos.
perhaps you'll create a directory for each user and, inside that, a directory for each post. if you name the directory with something like the date, time and title or a unique id like we've discussed for the blog posts, you can use photo.jpg and description.txt or something similar inside the directory.
another possibility is to have one directory for photos and another for descriptions, have a subdirectory in each for each user then have matching filenames inside those.
again, these could be subdivided by year, month or day, depending on the expected quantity of posts.
as you can see, sometimes the same considerations come up in terms of uniqueness, quantity and complexity. while other considerations like matching and format tracking (jpg, heic, webp, etc in this case) only apply in some scenarios.
from here on, we're going to assume __DIR__ is /www/httpdocs and today's date/time is november 15, 2173 at 19.16.50. if yours is something different (which it probably is), keep in mind that's only a placeholder for whatever your web root is.
php's built-in command for writing a text file is file_put_contents.
$fileTitle = "fruits";
$fileContents = "apples are my favorite fruit.\nespecially in pies!";
!file_put_contents(__DIR__ . "/posts/" . date("Y.m.d.H.i.s") . " " . $fileTitle . ".txt", $fileContents) && print_r("your post couldn't be saved.");generates a file /www/httpocs/posts/2173.11.15.19.16.50 fruits.txt with contents...
apples are my favorite fruit.
especially in pies!file_put_contents returns true if it successfully writes the file so your post couldn't be saved. only appears if something went wrong and you'll hopefully never see it.
as you might expect, php has a partner built-in function called file_get_contents that reads files.
$fileContents = file_get_contents(__DIR__ . "/posts/2173.11.15.19.16.50 fruits.txt");would get the file we just wrote as $fileContents.
file_get_contents has a more powerful ability, though. it can do far more than just read a text file from the server. it can grab files from the web if you need them.
$sfoForecast = file_get_contents("https://api.weather.gov/gridpoints/MTR/85,105/forecast");reads the national weather service forecast for san francisco into $sfoForecast, which you can then treat as a string in your program. this works for any plain get request.
there are many ways to separate information in text. in the case of a blog post, it might be as simple as making the first line the date and time, the second line the title and the remainder of the file the content of the post.
$fileName = __DIR__ . "/posts/32e4fcea-0d67-4d63-875c-81606e912fa1.txt";
if ($f = file_get_contents($fileName)) {
$f = explode("\n", $f);
isset($f[0]) && ($postDatetime = array_shift($f));
isset($f[0]) && ($postTitle = array_shift($f));
isset($f[0]) && ($postText = implode("\n", $f));
}first, we set the filename we're going to read from.
next, we read from the file. we're checking to make sure the file does, in fact, read properly before we try to do anything with that information. if this was inside a function, you could just return if the file doesn't read. in this case, however, to keep things simple, we just wrap the whole thing in an if block.
we use explode to split the file into an array by new line character (\n).
then we check to make sure there is a value in the first element of the array $f[0] and, if there is, remove it from the array and assign it to $postDatetime. we do the same thing for the next line as $postTitle.
finally, we check to make sure there's a value in the first element of the array then combine the rest of the array back together with \n between its elements and assign that to $postText.
if the data we're storing and retrieving is more of a table, we might use tabs or commas to separate it on a single row and separate the rows by new lines.
earlier, we used this file...
apple zucchini mint
peach broccoli cayenne pepper
pear cauliflower paprika
plum potato cumin
cherry turnip cinnamonwith tab and new line delimiters. each row is an entry of a fruit, a vegetable and a spice.
$fileName = __DIR__ . "/posts/e9e87f4b-5a04-4c4d-bbe0-1ce18b73b17f.txt";
if ($f = file_get_contents($fileName)) {
$f = explode("\n", $f);
$f = array_filter($f);
isset($f[0]) &&
array_walk($f, function (&$v) {
$v = explode("\t", $v);
});
}we set the filename, get the file and separate it by new lines the same way as before.
next, we use array_filter to remove blank entries in our array (blank lines, in this case).
then we use array_walk to go through each element in the array and apply a simple function to it. note that the & means the changes we make to $v are applied to the original variable ($f) instead of being kept inside the function, as we have already talked about.
much like in the earlier line, we just use explode to separate the input variable into an array using \t, tab. this returns a nested array...
[["apple", "zucchini", "mint"], ["peach", "broccoli", "cayenne pepper"], ["pear", "cauliflower", "paprika"], ["plum", "potato", "cumin"], ["cherry", "turnip", "cinnamon"]]this can feel like a lot of work for something php can already do for you, however. most of the time, instead of just storing your information as text files that you have to delimit and parse, you can use json.
json, not to be confused with the character from the friday the thirteenth movies, is javascript object notation. as the name implies, it was created for use with javascript as a way to store information. it has since become the de facto standard for small-quantity data storage on the internet, regardless of the language. what used to be stored in delimited text or xml files is almost certainly easier to implement and use in json. it's been around since the early 2000s and php has integrated it in its built-in functions since 2006. you'll see manual php versions of it, too, from before that -- i wrote one of them. but there's no need for that complexity anymore.
the main usefulness of json in php is that you can easily store php arrays as json and load them as if they'd never been anything else. yes, you can certainly just store an array to a file but, unless you want to encounter some massive potential security vulnerabilities, turning a file straight back into a php array is a much more involved process. json makes that simple.
the two main components of this storage method are the built-in functions json_encode and json_decode.
when we talked about functions, we created this array...
$people = [
[
"firstName" => "jasmine",
"lastName" => "park",
"email" => "[email protected]",
"height" => 174,
"weight" => 51,
"hairColor" => "blue",
"eyeColor" => "brown",
],
[
"firstName" => "katherine",
"lastName" => "law",
"email" => "[email protected]",
"height" => 157,
"weight" => 56,
"hairColor" => "black",
"eyeColor" => "brown",
],
[
"firstName" => "susan",
"lastName" => "yu",
"email" => "[email protected]",
"height" => 160.1,
"weight" => 60.4,
"hairColor" => "red",
"eyeColor" => "gray",
],
[
"firstName" => "jen",
"lastName" => "bull",
"email" => "[email protected]",
"height" => 165,
"weight" => 58,
"hairColor" => "blonde",
"eyeColor" => "green",
],
[
"firstName" => "lori",
"lastName" => "bratt",
"email" => "[email protected]",
"height" => 170.3,
"weight" => 67.2,
"hairColor" => "pink",
"eyeColor" => "blue",
],
];we can certainly output that array as a tab-delimited file...
jasmine park [email protected] 174 51 blue brown
katherine law [email protected] 157 56 black brown
susan yu [email protected] 160.1 60.4 red gray
jen bull [email protected] 165 58 blonde green
lori bratt [email protected] 170.3 67.2 pink bluebut that requires us to worry about special characters (what if there's a tab or new line in the data, for example) and making sure we remember the order of our information.
if, instead, we execute json_encode($people);, the result looks like...
[{"firstName":"jasmine","lastName":"park","email":"[email protected]","height":174,"weight":51,"hairColor":"blue","eyeColor":"brown"},{"firstName":"katherine","lastName":"law","email":"[email protected]","height":157,"weight":56,"hairColor":"black","eyeColor":"brown"},{"firstName":"susan","lastName":"yu","email":"[email protected]","height":160.1,"weight":60.4,"hairColor":"red","eyeColor":"gray"},{"firstName":"jen","lastName":"bull","email":"[email protected]","height":165,"weight":58,"hairColor":"blonde","eyeColor":"green"},{"firstName":"lori","lastName":"bratt","email":"[email protected]","height":170.3,"weight":67.2,"hairColor":"pink","eyeColor":"blue"}]we can make that readable by calling json_encode($people, JSON_PRETTY_PRINT);, giving us...
[
{
"firstName": "jasmine",
"lastName": "park",
"email": "[email protected]",
"height": 174,
"weight": 51,
"hairColor": "blue",
"eyeColor": "brown"
},
{
"firstName": "katherine",
"lastName": "law",
"email": "[email protected]",
"height": 157,
"weight": 56,
"hairColor": "black",
"eyeColor": "brown"
},
{
"firstName": "susan",
"lastName": "yu",
"email": "[email protected]",
"height": 160.1,
"weight": 60.4,
"hairColor": "red",
"eyeColor": "gray"
},
{
"firstName": "jen",
"lastName": "bull",
"email": "[email protected]",
"height": 165,
"weight": 58,
"hairColor": "blonde",
"eyeColor": "green"
},
{
"firstName": "lori",
"lastName": "bratt",
"email": "[email protected]",
"height": 170.3,
"weight": 67.2,
"hairColor": "pink",
"eyeColor": "blue"
}
]there's certainly no reason to use that unless people are going to have to edit the json manually because it just takes up more space to store your data. but it gives us a clear demonstration of the benefit. it knows the difference between strings and numbers and attaches the name of the information to the information itself. that makes it flexible. if some people are missing information or if new people are added with extra, nothing has to change about the structure.
is this the ideal data storage for billions of records in a single table? absolutely not. but, if the goal is to be able to read whole blocks of information and treat it as native data inside your program, this is probably the best solution.
in practice, it might be implemented like this...
function save($f, array $data)
{
return count($data) && ((!file_exists($f) && is_writable(dirname($f))) || (file_exists($f) && is_writable($f))) && file_put_contents($f, json_encode($data));
}
function load($f)
{
return is_readable($f) && ($data = file_get_contents($f)) ? json_decode($data, true) : false;
}save takes a filename and an array of data to save in that file.
first, we make sure the array $data actually contains data. if not, there's no point in continuing so we return false.
second, we have a pair of possibilities. either the file doesn't already exist but the directory is writable so we can create the file or the file does exist and the file is writable so we can change it. if neither is the case (the file exists and we can't change it or the directory isn't writable), we return false.
third, we try to write the file $f with $data encoded as json. if that succeeds, we return true. otherwise, we return false.
load does the reverse. it takes a filename and checks if it's readable then reads it into a variable $data. if either of those fails, it returns false. if it succeeds, it returns $data as an array, decoded from json. the true parameter for json_decode tells php to return it as an array instead of an object.
when we discussed object-oriented programming, we mentioned the object type, which is functionally just an array with slightly different treatment. while there are use-cases for the object type in php, they are very few by comparison with arrays so the json_decode function will almost certainly pass this true parameter when you're using it.
if we take an array and pass it to save then pass the same filename to load, the result will be the same array. practically-speaking, if you're using a set of functions like this, you may either need another pair of functions for non-json data (like markdown or plain-text) or another parameter to differentiate the desired results.
my preference in cases like this is to create a combined function for both types of results and create secondary functions like loadJson and loadText that call them with the desired parameters. either solution, however, works.
most of the time, if you're dealing with binary files like photos or videos, you'll be calling them directly with their urls and this is definitely best practice. having to load files from the filesystem using php then serve them instead of allowing the web server to just send them out is almost always a bad idea because it uses a lot of unnecessary server resources for no change in output. there are special cases where it's necessary, however.
these usually involve changing the file in some way -- adding a custom download watermark to an image telling which user accessed it or creating a zip with specific files then sending it to a user and removing it after it's been downloaded.
to load a file from the filesystem, you can use file_get_data, just like for plain-text. if the file is very large, however, this can fail. so you can use fopen. note that it is important to pass "rb" to fopen to tell it to read as binary.
function readBinary($f)
{
return is_readable($f) && ($file = fopen($f, "rb")) && ($binary = fread($file, filesize($f))) && fclose($file) ? $binary : false;
}first, we check to make sure $f is a readable file.
then we create a file handle, meaning we ask php to open the file -- rb meaning to read as a binary file.
if it opens, we set variable $binary to be the contents of the file from the beginning to filesize($f), the total size of the file. this function allows us to specify how much of the file to read if we want to do it in pieces, which can be useful for extremely large files.
if that's successful, we close the file. this will eventually happen automatically but it's good practice and much more efficient to do it as soon as we're finished with the file to save server resources.
if all that succeeds, we return the variable $binary with the contents of the file. otherwise, we return false.
it's important to remember, as you're trying these functions for yourself, that these all depend on the permissions we already discussed allowing the user php is running as to access the files. that's not necessarily the user you're logged in as to edit the php. if things aren't working as you expect, it might not be your code. check the ownership of the files you're trying to read and write and compare that with the user php is running as. you might spend hours troubleshooting a coding error only to discover the code was perfect and the file simply wasn't accessible to the code. if you think i've mentioned this too many times or belabored this point, check any programming forum for any language and you'll discover thousands of recent posts where people are making exactly this mistake.
one quick way to check is to try to read the php file itself. if the file is running, it must be readable by the web server's user...
print_r("my php file...\n\n" . file_get_contents(__FILE__));this is not helpful in terms of producing usable results. it will, however, tell you if your problem is permissions-based. this will work with __FILE__if your php is correctly configured. if it works with __FILE__ but not with the file you're trying to access elsewhere on the server, you know it's not a code error but a permissions or access one.