File System in Node.js Part 3: Working Asynchronously
March 18, 2021
The Sync suffix included in the functions such as fs.existsSync and fs.mkdirSync indicates that these functions are synchronous (sync) and they have asynchronous (async) versions. Synchronous operation is an operation where we have to wait for it to finish before the next operation can start. This is pretty much how a lot of things we have seen so far are working. For example, when we performed an operation on one line and wanted to console.log the result of that operation on the next one, the process waits for the first operation to finish. That is how we are able to display the result of the operation on the next line.
const result = 10 + 10;
console.log(result);
There are cases when this linear order of operations is not desirable. Sometimes we might want to execute a statement but not care about the results immediately after. This is especially relevant when working with processes that take a longer time to perform. Input and output operations such as creating, reading, or writing files and folders are usually of this nature. They tend to take much longer compared to operations that happen in the system memory or CPU. We might not want to block the system from doing any further operations by synchronously performing the slow processes. This is why we have asynchronous versions of these potentially costly operations. When we execute an asynchronous function, the program doesn't wait around for that operation's result but continues to execute the following lines.
In Node.js, there are two ways to perform async operations. One of them is using callback functions, and the other one is using Promises.
Let's look at how we asynchronously perform file system operations using callback functions.
Creating a folder asynchronously
Let's asynchronously create a folder.
const fs = require("fs");
const folderPath = "myFolder";
const callback = (err) => {
console.log(err);
};
fs.mkdir(folderPath, callback);
Here we are using the mkdir function instead of the mkdirSync function to create the folder in an async way. mkdir function takes two arguments. The first one is the relative path for the desired folder, and the second argument is a callback function.
A callback function is just a descriptive name for a function that gets passed into another function. The function that receives the callback function chooses when it wants to execute the function. Here is a straightforward example of a callback function.
const someOperation = () => {
console.log("this is a function that represents some operation");
};
const logStartAndEndForOperation = (callbackFunction) => {
console.log("start of operation");
callbackFunction();
console.log("end of operation");
};
logStartAndEndForOperation(someOperation);
In this example, someOperation is a callback function in relation to logStartAndEndForOperation function since it is passed as an argument to that function. logStartAndEndForOperation function takes a callback function as an argument and determines when to call that given function.
mkdir function also takes a callback function as an argument and calls that function when the directory creation operation is finished. mkdir function calls the callback function with a single argument, which is the error object with information about the errors that might have occurred during the directory creation. If there are no errors, then it would call the callback function with a null. Here is an example to illustrate calling the callback function with an argument.
const someOperation = () => {
console.log("this is a function that represents some operation");
};
const logStartAndEndForOperation = (callbackFunction) => {
const importantValue = 42;
console.log("start of operation");
callbackFunction(importantValue);
console.log("end of operation");
};
logStartAndEndForOperation(someOperation);
I have adjusted the previous example slightly to call the callback function with a numeric value of 42.
Note that the someOperation function doesn't actually expect any arguments in our example, but that doesn't mean we can't pass arguments to it. It is just that the function wouldn't be using those arguments.
Going back to the mkdir function, the error object that gets passed into the callback function carries information about what went wrong during the process. Receiving an error object instead of a null value means that the folder didn't get created for some reason.
const fs = require("fs");
const folderPath = "myFolder";
const callback = (err) => {
if (err) {
if (err.code === "EEXIST") {
console.log("Folder already exists");
} else {
console.log("Something went wrong");
}
} else {
console.log("Success!");
}
};
fs.mkdir(folderPath, callback);
If we console.log the error object to the screen, we would see that it would be null if the directory is created successfully. But it would be an object with some properties if there is an error condition like the same directory name already existing. One of these properties is the code property that has a code for the error reason. For the case where the directory already exists, the code would be equivalent to EEXIST. We can use this code to console.log a message that informs the user that the folder already exists. If there is an error object, but the code is different, we can simply console.log a generic error message that communicates something went wrong for a simple error handling mechanism. If there are no errors, we can assume that the folder is successfully created and displaying a success message.
In this example, we don't need to check to see if the folder exists anymore since we would get an error anyway if it does and handle the situation accordingly.
When using callback functions, error handling is a bit different than compared to working synchronously. In a synchronous scenario, we would be using try...catch statements to handle errors. In callback functions, we are using the error object that we receive to handle the errors.
How do we know if this function is asynchronous? We can verify that by placing a console.log statement right after the mkdir call. We would notice that the console.log statement gets executed before the callback function. This is because the input-output operations such as creating a folder take much longer to complete than merely logging a value to the screen.
const fs = require("fs");
const folderPath = "myFolder";
const callback = (err) => {
if (err) {
if (err.code === "EEXIST") {
console.log("Folder already exists");
} else {
console.log("Something went wrong");
}
} else {
console.log("Success!");
}
};
fs.mkdir(folderPath, callback);
console.log("End of program");
We might see callback functions not separately declared as we did here but defined inline where they are being passed into the function. That might make sense given that this is not a function that we are planning on reusing but want to pass as an argument to another function call. Having an inline callback function would look something like this:
const fs = require("fs");
const folderPath = "myFolder";
fs.mkdir(folderPath, (err) => {
if (err) {
if (err.code === "EEXIST") {
console.log("Folder already exists");
} else {
console.log("Something went wrong");
}
} else {
console.log("Success!");
}
});
console.log("End of program");
It is not an incident that the first argument for the callback function for mkdir is the err object. That is a pretty well-established convention in Node.js. Let's take a look at other asynchronous functions to understand the callback function arguments a bit better.
Reading and Writing files in Node.js
To be able to write to a file asynchronously, we will be using the writeFile function. Here is how we do it:
const fs = require("fs");
const filePath = "temp.txt";
const fileData = "hello world";
fs.writeFile(filePath, fileData, (err) => {
if (err) {
console.log(err);
}
});
We are using the async version of writeFile. Sync version for the same functions exists under the name writeFileSync.
This code creates a file called temp.txt in the folder where we call the script from and writes the data hello world into that file. If we were to execute this script again, it would work without throwing an error, unlike the mkdir function. It doesn't complain when the target file already exists; it just overwrites it with new data. And just like we learned before, the first argument for the callback function is the error object.
Let's look at how to read files.
const fs = require("fs");
const filePath = "temp.txt";
fs.readFile(filePath, "utf8", (err, data) => {
if (err) {
console.log(err);
} else {
console.log(data);
}
});
This code will read the content of the temp.txt file granted that it exists. We would receive an error message with the error code ENOENT in case the file does not exist. We can update our code to handle that error message.
const fs = require("fs");
const filePath = "temp.txt";
fs.readFile(filePath, "utf8", (err, data) => {
if (err) {
if (err.code === "ENOENT") {
console.log("File does not exist");
} else {
console.log("Something went wrong");
}
} else {
console.log(data);
}
});
What if we wanted to get the content of the file that we read and pass it to another function for further processing? Doing so was very straightforward when we were working with synchronous functions. Here is what that would look like if the function is synchronous.
const fs = require("fs");
const filePath = "temp.txt";
const data = fs.readFileSync(filePath, "utf8");
console.log(data);
This would not be possible to do when working with asynchronous functions. We can't just execute an async function and expect to capture the result with a variable assignment since the result of that execution doesn't become available until later. We need to do any further processing on the result inside the callback function.
Say we have a function that prints out some statistical information about the given string, like the string's length and the number of words in it.
const fs = require("fs");
const getTextData = (text) => {
const numChars = text.length;
const numWords = text.split(" ").length;
console.log(`Given text has ${numChars} characters and ${numWords} words.`);
};
const main = () => {
const filePath = "temp.txt";
fs.readFile(filePath, "utf8", (err, data) => {
if (err) {
console.log("Something went wrong");
console.log(err);
return;
} else {
getTextData(data);
}
});
};
main();
As we can see, if we wanted to operate on the results of readFile function, we would have to do that from inside the callback function of readFile.
fs.readFile(filePath, "utf8", (err, data) => {
if (err) {
console.log("Something went wrong");
console.log(err);
return;
} else {
getTextData(data);
}
});
Choosing in Between Synchronous and Asynchronous Functions
Now that we know how to work with synchronous and asynchronous functions you wonder how to choose between two methods. As you might have noticed, synchronous functions are much easier to work with since it results in a linear code that runs from top-to-bottom. In comparison, asynchronous functions require more lines of code to write and are more complicated due to their non-linear nature.
Having said that, there are substantial performance benefits for using asynchronous operations. It makes sense to use synchronous functions if the performance is not a consideration. Otherwise, asynchronous functions are the way to go.
We have only seen using callback functions for asynchronous operations. There are also Promises and async/await to handle asynchronous operations that we will be exploring later.
Summary
In this section, we have learned about the difference between synchronous and asynchronous operations. We have used asynchronous file system operations such as fs.mkdir, fs.writeFile, fs.readFile.
We have also learned how to choose in between synchronous and asynchronous operations. If performance is a concern, we should choose to work with asynchronous operations; otherwise, synchronous operations are also fine.
Find me on Social Media