The Thinks I Think
I had the opportunity to give a talk yesterday. And it was more significant than I realized at first.
This is a reading of the blog post here (https://brainofshawn.com/2022/05/20/significance/)
Last night I had the opportunity to speak for Linux Dominicana, which is the Linux Users Group in the Dominican Republic. I was approached several months ago by a man who has since become a good friend, asking if I would give a webinar-based talk. As most people reading this know, I’m currently trying to produce more and more content, and hopefully reach more and more people with it. So even though I had concerns over cultural errors I might make, I agreed to do the talk.
Spoiler: I’m very glad I did.
My whole personal “brand” thing about being kind, assuming the best in others, admitting when I’m wrong, etc., is not so much a “brand” as genuinely who I want to be as a human. And so in the months leading up to the talk, a bunch of the leaders joined me in a group chat so I could ask them an annoying amount of questions. I was not worried about embarrassing myself (clearly, based on most of my public interactions, I have no problem looking silly). But I was very concerned that I would make an offensive comment, or make an assumption out of ignorance that would hurt the feelings those attending. I gave my rough talk outline (it was about Linux as a vehicle to a career), and I asked awkward questions about job opportunities, difficulties with language barriers, salary inequalities, and how things “work” in the Dominican Republic.
This group of folks were so patient, and so kind, my gratitude doesn’t seem like enough payment for all the education they provided me. After a couple months of trying to learn Spanish, it was clear I wouldn’t be anywhere close to capable of conversation in their native language, and still they were nothing but patient with my litany of questions. Honestly, I was probably rather annoying. And even though I learned a lot about Dominican people, and even about Latin America in general — the upcoming talk was more distressing than pretty much any other talk I’ve given. (OK, the Ohio LinuxFest keynote address where I lost my entire presentation the night before was pretty stressful, but this was a different sort of stress.)
See… I wasn’t sure I’d be relatable. Don’t get me wrong, human stories are about humans, and in general we can empathize if we try. But would my life experiences translate (literally and figuratively) to another culture? I normally weave sarcasm and self deprecation into my talks, but sarcasm is often hard to pick up in your native language, so my standard go-to would probably fall flat. Plus, my talk wasn’t really about anything technical. It was a story about how I found my passion, and how those passions helped me in my own career. It was a very “soft” talk for a group of hardened IT professionals.
How did it go? I think the talk went fine. (That link will start the talk when it switches to English) It wasn’t earth-shattering. It was a story-based look at my career, with a few pointers for finding passion of your own. And an awkward section about the unfair importance of speaking English. Again, it wasn’t a bad talk. But it wasn’t amazing by any stretch of the definition.
But it was significant.
What I didn’t mention earlier is that this was the first time they’ve had one of their presentations in English. I was even one of the first (maybe the actual first) presenter without a tie to the Dominican Republic at all. Heck, I’m so white I’m almost clear! They did this on purpose, but I didn’t realize it was that out of the ordinary until I was chatting with the group of leaders the day before the talk. Whether or not my talk went well, they assured me it was ground-breaking. Assuming the attendees enjoyed the talk, it probably means they will have future speakers from other places with different views and different insights. And I got to be the first person to open that door. I’m still so very humbled.
I hope my mediocre talk, where I tried to speak more slowly than normal (I talk fast when I get excited) was enough to tear down a few cultural walls. The kindness of commenters, saying things in English so I could understand it, was really quite touching. This group of Dominicans will always have a special place in my heart. And once winter sets in, I might find out if their offers of helping me get the most out of a trip to the DR were genuine. And to be honest, I already know they were. Because along with being the first English-only speaker to their group, I also made a pretty great group of friends. And that. That is how we change the world.
I have a 48TB NAS in my basement. Granted, thanks to RAID6 I only (only!) have 36TB of usable space, but still, I assumed that would last me forever. Thanks to ripping DVDs and Blurays, I was so very, very wrong. Rather than spend a few thousand dollars on a new NAS, however, I decided to host my files in the cloud. The storage is unlimited for $12/month, and after 6 months or so, I can tell you it’s a viable alternative to local storage. Plus, it mounts exactly like a local NAS!
There are plenty of cloud-based storage solutions available, but they are all either very limited in storage space, or very expensive per GB. There is one solution, however, that provides unlimited storage for a set monthly price. Google Drive.
Officially, in order to get unlimited storage, you must get a Business Gsuite with 5 users. Each user is $12/month, and so you’d have to pay $60/month to get unlimited storage. Honestly, $60 a month for that much space is still insanely affordable — but if you open a Business Gsuite account with a single user (so only $12/mo), you still get unlimited storage. It might seem like an error Google would quickly fix, but it’s been that way for years. I’m currently using more than 40TB of space on my Google Drive, and only have a single user on my Gsuite for Business.
Google Drive is nice, but let’s be honest, no one wants to use their web interface as bulk storage. It’s clumsy, slow, and as much as I love Google, the organization is confusing at best. Google does provide “Google Drive Stream”, but due to local caching, you still need local storage or you get “not enough space” errors.
Thankfully, Rclone makes direct access to Google Drive seamless. It allows you to create access via keypair (no annoying logging in all the time), and even lets you mount your remote share on your local filesystem. And in true Steve Jobs “one more thing” fashion, it also allows you to encrypt files and directories in real time, so your privacy is protected even if your data is stored on someone else’s hard drive. It’s seriously amazing. And Rclone? Open Source and totally free!
Rclone is in most Linux distributions, and even has Windows and OSX versions available that all work similarly. In this video, I show you how to quickly set up a share and mount it on the local filesystem. If it seems too good to be true, yeah, I get that. But I’ve been using it for months and I’ve been more than impressed. It’s been reliable, and robust enough to support a handful of users reading and writing at the same time.
You can do a bit of extra work to create your own application API, which will make the performance more robust. It doesn’t cost any extra money, but it’s admittedly a bit of confusing clicking.
You probably know I create training for a living. I have more in-depth training on rclone over at CBT Nuggets. If you’re already a subscriber, you can go to https://snar.co/cbt-rclone to get to the skill directly.
If you’re not yet a subscriber at CBT Nuggets, you can see my Everything Linux course overview, which includes my rclone skill and many others. Feel free to sign up for a free trial if you want to view my training. https://snar.co/cbt-everythinglinux
(This isn’t a creepy bait and switch — the free video above really does walk you through the process. There’s just more capability if you’re interested in diving in deeper, and want to check out my professional DayJob training!)
This article covers some grep and regex basics.
There are generally two types of coffee drinkers. The first type buys a can of pre-ground beans and uses the included scoop to make their automatic drip coffee in the morning. The second type picks single-origin beans from various parts of the world, accepts only beans that have been roasted within the past week and grinds those beans with a conical burr grinder moments before brewing in any number of complicated methods. Text searching is a bit like that.
For most things on the command line, people think of *.* or *.txt and are happy to use file globbing to select the files they want. When it comes to grepping a log file, however, you need to get a little fancier. The confusing part is when the syntax of globbing and regex overlap. Thankfully, it’s not hard to figure out when to use which construct.
The command shell uses globbing for filename completion. If you type something like ls *.txt, you’ll get a list of all the files that end in .txt in the current directory. If you do ls R*.txt, you’ll get all the files that start with capital R and have the .txt extension. The asterisk is a wild card that lets you quickly filter which files you mean.
You also can use a question mark in globbing if you want to specify a single character. So, typing ls read??.txt will list readme.txt, but not read.txt. That’s different from ls read*.txt, which will match both readme.txt and read.txt, because the asterisk means “zero or more characters” in the file glob.
Here’s the easy way to remember if you’re using globbing (which is very simple) vs. regular expressions: globbing is done to filenames by the shell, and regex is used for searching text. The only frustrating exception to this is that sometimes the shell is too smart and conveniently does globbing when you don’t want it to—for example:
grep file* README.TXT
In most cases, this will search the file README.TXT looking for the regular expression file*, which is what you normally want. But if there happens to be a file in the current folder that matches the file* glob (let’s say filename.txt), the shell will assume you meant to pass that to grep, and so grep actually will see:
grep filename.txt README.TXT
Gee, thank you so much Mr. Shell, but that’s not what I wanted to do. For that reason, I recommend always using quotation marks when using grep. 99% of the time you won’t get an accidental glob match, but that 1% can be infuriating. So when using grep, this is much safer:
grep "file*" README.TXT
Because even if there is a filename.txt, the shell won’t substitute it automatically.
So, globs are for filenames, and regex is for searching text. That’s the first thing to understand. The next thing is to realize that similar syntax means different things.
I don’t want this article to become a super in-depth piece on regex; rather, I want you to understand simple regex, especially as it conflicts with blobbing. Table 1 shows a few of the most commonly confused symbols and what they mean in each case.
| Special Character | Meaning in Globs | Meaning in Regex |
| * | zero or more characters | zero or more of the character it follows |
| ? | single occurrence of any character | zero or one of the character it follows but not more than 1 |
| . | literal “.” character | any single character |
To add insult to injury, you might be thinking about globs when you use grep, but just because you get the expected results doesn’t mean you got the results for the correct reason. Let me try to explain. Here is a text file called filename.doc:
The fast dog is fast.
The faster dogs are faster.
A sick dog should see a dogdoc.
This file is filename.doc
If you type:
grep "fast*" filename.doc
The first two lines will match. Whether you’re thinking globs or regex, that makes sense. But if you type:
grep "dogs*" filename.doc
The first three lines will match, but if you’re thinking in globs, that doesn’t make sense. Since grep uses regular expressions (regex) when searching files, the asterisk means “zero or more occurrences of the previous character”, so in the second example, it matches dog and dogs, because having zero “s” characters matches the regex.
And let’s say you typed this:
grep "*.doc" filename.doc
This will match the last two lines. The asterisk doesn’t actually do anything in this command, because it’s not following any character. The dot in regex means “any character”, so it will match the “.doc”, but it also will match “gdoc” in “dogdoc”, so both lines match.
The moral of the story is that grep never uses globbing. The only exception is when the shell does globbing before passing the command on to grep, which is why it’s always a good idea to use quotation marks around the regular expression you are trying to grep for.
fgrep to Avoid RegexIf you don’t want the power of regex, it can be very frustrating. This is especially true if you’re actually looking for some of the special characters in a bunch of text. You can use the fgrep command (or grep -F, which is the same thing) in order to skip any regex substitutions. Using fgrep, you’ll search for exactly what you type, even if they are special characters. Here is a text file called file.txt:
I really hate regex.
All those stupid $, {}, and \ stuff ticks me off.
Why can't text be text?
If you try to use regular grep like this:
grep "$," file.txt
you’ll get no results. That’s because the “$” is a special character (more on that in a bit). If you’d like to grep for special characters without escaping them, or knowing the regex code to get what you want, this will work fine:
grep -F "$," file.txt
And, grep will return the second line of the text file because it matches the literal characters. It’s possible to build a regex query to search for special characters, but it can become complicated quickly. Plus, fgrep is much, much faster on a large text file.
Okay, now that you know when to use globbing and when to use regular expressions, let’s look at a bit of regex that can make grepping much more useful. I find myself using the caret and dollar sign symbols in regex fairly often. Caret means “at the beginning of the line”, and dollar sign means “at the end of the line”. I used to mix them up, so my silly method to remember is that a farmer has to plant carrots at the beginning of the season in order to sell them for dollars at the end of the season. It’s silly, but it works for me!
Here’s a sample text file named file.txt:
chickens eat corn
corn rarely eats chickens
people eat chickens and corn
chickens rarely eat people
If you were to type:
grep "chickens" file.txt
you will get all four lines returned, because “chickens” is in each line. But if you add some regex to the mix:
grep "^chickens" file.txt
you’ll get both the first and fourth line returned, because the word “chickens” is at the beginning of those lines. If you type:
grep "corn$" file.txt
you will see the first and third lines, because they both end with “corn”. However, if you type:
grep "^chickens.*corn$" file.txt
you will get only the first line, because it is the only one that begins with chickens and ends with corn. This example might look confusing, but there are three regular expressions that build the search. Let’s look at each of them.
First, ^chickens means the line must start with chickens.
Second, .* means zero or more of any character, because remember, the dot means any character, and the asterisk means zero or more of the previous character.
Third, corn$ means the line must end with corn.
When you’re building regular expressions, you just mush them all together like that in a long string. It can become confusing, but if you break down each piece, it makes sense. In order for the entire regular expression to match, all of the pieces must match. That’s why only the first line matches the example regex statement.
A handful of other common regex characters are useful when grepping text files. Remember just to mush them together to form the entire regular expression:
\ — the backslash negates the “special-ness” of special characters, which means you actually can search for them with regex. For example, \$ will search for the $ character, instead of looking for the end of a line.\s — this construct means “whitespace”, which can be a space or spaces, tabs or newline characters. To find the word pickle surrounded by whitespace, you could search for \spickle\s, and that will find “pickle” but not “pickles”..* — this is really just a specific use of the asterisk, but it’s a very common combination, so I mention it here. It basically means “zero or more of any characters”, which is what was used in the corn/chicken example above.| — this means “or” in regex. So hi|hello will match either “hi” or “hello”. It’s often used in parentheses to separate it from other parts of the regular expression. For example, (F|f)rankfurter will search for the word frankfurter, whether or not it’s capitalized.[] — brackets are another way to specify “or” options, but they support ranges. So the regex [Ff]rankfurter is the same as the above example. Brackets support ranges though, so ^[A-Z] will match any line that starts with a capital letter. It also supports numbers, so [0-9]$ will match any line that ends in a digit.You can do far more complicated things with regular expressions. These basic building blocks are usually enough to get the sort of text you need out of a log file. If you want to learn more, by all means, either do some googling on regex, or get a book explaining all the nitty-gritty goodness. If you want me to write more about it, drop a comment and let me know.
I really, really encourage you to practice using regex. The best way to learn is to do, so make a few text files and see if the regex statements you create give you the results you expect. Thankfully, grep highlights the “match” it finds in the line it returns. That means if you’re getting more results than you expect, you’ll see why the regex matched more than you expected, because grep will show you.
The most important thing to remember is that grep doesn’t do globbing—that wild-card stuff is for filenames on the shell only. Even if globbing with grep seems to work, it’s probably just coincidence (look back at the dog/dogs example here if you don’t know what I’m talking about). Have fun grepping!
Roles are the most complicated and yet simplest aspect of Ansible to learn.
I’ve mentioned before that Ansible’s ad-hoc mode often is overlooked as just a way to learn how to use Ansible. I couldn’t disagree with that mentality any more fervently than I already do. Ad-hoc mode is actually what I tend to use most often on a day-to-day basis. That said, using playbooks and roles are very powerful ways to utilize Ansible’s abilities. In fact, when most people think of Ansible, they tend to think of the roles feature, because it’s the way most Ansible code is shared. So first, it’s important to understand the relationship between ad-hoc mode, playbooks and roles.
This is a bit of a review, but it’s easy to forget once you start creating playbooks. Ad-hoc mode is simply a one-liner that uses an Ansible module to accomplish a given task on a set of computers. Something like:
ansible cadlab -b -m yum -a "name=vim state=latest"
will install vim on every computer in the cadlab group. The -b signals to elevate privilege (“become” root), the -m means to use the yum module, and the -a says what actions to take. In this case, it’s installing the latest version of vim.
Usually when I use ad-hoc mode to install packages, I’ll follow up with something like this:
ansible cadlab -b -m service -a "name=httpd state=started
↪enabled=yes"
That one-liner will make sure that the httpd service is running and set to start on boot automatically (the latter is what “enabled” means). Like I said at the beginning, I most often use Ansible’s ad-hoc mode on a day-to-day basis. When a new rollout or upgrade needs to happen though, that’s when it makes sense to create a playbook, which is a text file that contains a bunch of Ansible commands.
I described playbooks in my last article. They are YAML- (Yet Another Markup Language) formatted text files that contain a list of things for Ansible to accomplish. For example, to install Apache on a lab full of computers, you’d create a file something like this:
---
- hosts: cadlab
tasks:
- name: install apache2 on CentOS
yum: name=httpd state=latest
notify: start httpd
ignore_errors: yes
- name: install apache2 on Ubuntu
apt: update_cache=yes name=apache2 state=latest
notify: start apache2
ignore_errors: yes
handlers:
- name: start httpd
service: name=httpd enable=yes state=started
- name: start apache2
service: name=apache2 enable=yes state=started
Mind you, this isn’t the most elegant playbook. It contains a single play that tries to install httpd with yum and apache2 with apt. If the lab is a mix of CentOS and Ubuntu machines, one or the other of the installation methods will fail. That’s why the ignore_errors command is in each task. Otherwise, Ansible would quit when it encountered an error. Again, this method works, but it’s not pretty. It would be much better to create conditional statements that would allow for a graceful exit on incompatible platforms. In fact, playbooks that are more complex and do more things tend to evolve into a “role” in Ansible.
Roles aren’t really a mode of operation. Actually, roles are an integral part of playbooks. Just like a playbook can have tasks, variables and handlers, they can also have roles. Quite simply, roles are just a way to organize the various components referenced in playbooks. It starts with a folder layout:
roles/
webserver/
tasks/
main.yml
handlers/
main.yml
vars/
main.yml
templates/
index.html.j2
httpd.conf.j2
files/
ntp.conf
Ansible looks for a roles folder in the current directory, but also in a system-wide location like /etc/ansible/roles, so you can store your roles to keep them organized and out of your home folder. The advantage of using roles is that your playbooks can look as simple as this:
---
- hosts: cadlab
roles:
- webserver
And then the “webserver” role will be applied to the group “cadlab” without needing to type any more information inside your playbook. When a role is specified, Ansible looks for a folder matching the name “webserver” inside your roles folder (in the current directory or the system-wide directory). It then will execute the tasks inside webserver/tasks/main.yml. Any handlers mentioned in that playbook will be searched for automatically in webserver/handlers/main.yml. Also, any time files are referenced by a template module or file/copy module, the path doesn’t need to be specified. Ansible automatically will look inside webserver/files/ or /webserver/templates/ for the files.
Basically, using roles will save you lots of path declarations and include statements. That might seem like a simple thing, but the organization creates a standard that not only makes it easy to figure out what a role does, but also makes it easy to share your code with others. If you always know any files must be stored in roles/rolename/files/, it means you can share a “role” with others and they’ll know exactly what to do with it—namely, just plop it in their own roles folder and start using it.
One of the best aspects of current DevOps tools like Chef, Puppet and Ansible is that there is a community of people willing to share their hard work. On a small scale, roles are a great way to share with your coworkers, especially if you have roles that are customized specifically for your environment. Since many of environments are similar, roles can be shared with an even wider audience—and that’s where Ansible Galaxy comes into play.
I’ll be honest, part of the draw for me with Ansible is the sci-fi theme in the naming convention. I know I’m a bit silly in that regard, but just naming something Ansible or Ansible Galaxy gets my attention. This might be one of those “built by nerds, for nerds” sort of things. I’m completely okay with that. If you head over to the Galaxy site, you’ll find the online repository for shared roles—and there are a ton.
For simply downloading and using other people’s roles, you don’t need any sort of account on Ansible Galaxy. You can search on the website by going to Galaxy and clicking “Browse Roles” on the left side of the page (Figure 1). There are more than 13,000 roles currently uploaded to Ansible Galaxy, so I highly recommend taking advantage of the search feature! In Figure 2, you’ll see I’ve searched for “apache” and sorted by “downloads” in order to find the most popular roles.
Figure 1. Click that link to browse and search for roles.
Figure 2. Jeff Geerling’s roles are always worth checking out.
Many of the standard roles you’ll find that are very popular are written by Jeff Geerling, whose user name is geerlingguy. He’s an Ansible developer who has written at least one Ansible book that I’ve read and possibly others. He shares his roles, and I encourage you to check them out—not only for using them, but also for seeing how he codes around issues like conditionally choosing the correct module for a given distribution and things like that. You can click on the role name and see all the code involved. You might notice that if you want to examine the code, you need to click on the GitHub link. That’s one of the genius moves of Ansible Galaxy—all roles are stored on a user’s GitHub page as opposed to an Ansible Galaxy server. Since most developers keep their code on GitHub, they don’t need to remember to upload to Ansible Galaxy as well.
Incidentally, if you ever desire to share your own Ansible roles, you’ll need to use a GitHub user name to upload them, because again, roles are all stored on GitHub. But that’s getting ahead of things; first you need to learn how to use roles in your environment.
It’s certainly possible to download an entire repository and then unzip the contents into your roles folder. Since they’re just text files and structured folders, there’s not really anything wrong with doing it that way. It’s just far less convenient than using the tools built in to Ansible.
There is a search mechanism on the Ansible command line for searching the Ansible Galaxy site, but in order to find a role I want to use, I generally go to the website and find it, then use the command-line tools to download and install it. Here’s an example of Jeff Geerling’s “apache” role. In order to use Ansible to download a role, you need to do this:
sudo ansible-galaxy install geerlingguy.apache
Notice two things. First, you need to execute this command with root privilege. That’s because the ansible-galaxy command will install roles in your system-wide roles folder, which isn’t writable (by default) by your regular user account. Second, take note of the format of roles named on Ansible Galaxy. The format is username.rolename, so in this case, geerlingguy.apache, which is also how you reference the role inside your playbooks.
If you want to see roles listed with the correct format, you can use ansible-galaxy‘s search command, but like I said, I find it less than useful because it doesn’t sort by popularity. In fact, I can’t figure out what it sorts by at all. The only time I use the command-line search feature is if I also use grep to narrow down roles by a single person. Anyway, Figure 3 shows what the results of ansible-galaxy search look like. Notice the username.rolename format.
Figure 3. I love the command line, but these search results are frustrating.
Once you install a role, it is immediately available for you to use in your own playbooks, because it’s installed in the system-wide roles folder. In my case, that’s /etc/ansible/roles (Figure 4). So now, if I create a playbook like this:
---
- hosts: cadlab
roles:
- geerlingguy.apache
Apache will be installed on all my cadlab computers, regardless of what distribution they’re using. If you want to see how the role (which is just a bunch of tasks, handlers and so forth) works, just pick through the folder structure inside /etc/ansible/roles/geerlingguy.apache/. It’s all right there for you to use or modify.
Figure 4. Easy Peasy, Lemon Squeezy
There’s really no magic here, since you easily can create a roles folder and then create your own roles manually inside it, but ansible-galaxy does give you a shortcut by creating a skeleton role for you. Make sure you have a roles folder, then just type:
ansible-galaxy init roles/rolename
and you’ll end up with a nicely created folder structure for your new role. It doesn’t do anything magical, but as someone who has misspelled “Templates” before, I can tell you it will save you a lot of frustration if you have clumsy fingers like me.
If you get to the point where you want to share you roles on Ansible Galaxy, it’s fairly easy to do. Make sure you have your role on GitHub (using git is beyond the scope of this article, but using git and GitHub is a great way to keep track of your code anyway). Once you have your roles on GitHub, you can use ansible-galaxy to “import” them into the publicly searchable Ansible Galaxy site. You first need to authenticate:
ansible-galaxy login
Before you try to log in with the command-line tool, be sure you’ve visited the Ansible Galaxy website and logged in with your GitHub account. You can see in Figure 5 that at first I was unable to log in. Then I logged in on the website, and after that, I was able to log in with the command-line tool successfully.
Figure 5. It drove me nuts trying to figure out why I couldn’t authenticate.
Once you’re logged in, you can add your role by typing:
ansible-galaxy import githubusername githubreponame
The process takes a while, so you can add the -no-wait option if you want, and the role will be imported in the background. I really don’t recommend doing this until you have created roles worth sharing. Keep in mind, there are more than 13,000 roles on Ansible Galaxy, so there are many “re-inventions of the wheel” happening.
Well, it’s taken me four articles, but I think if you’ve been following along, you should be to the point where you can take it from here. Playbooks and roles are usually where people focus their attention in Ansible, but I also encourage you to take advantage of ad-hoc mode for day-to-day maintenance tasks. Ansible in some ways is just another DevOps configuration management tool, but for me, it feels the most like the traditional problem-solving solution that I used Bash scripts to accomplish for decades. Perhaps I just like Ansible because it thinks the same way I do. Regardless of your motivation, I encourage you to learn Ansible enough so you can determine whether it fits into your workflow as well as it fits into mine.
If you’d like more direct training on Ansible (and other stuff) from yours truly, visit me at my DayJob as a trainer for CBT Nuggets. You can get a full week free if you head over to https://cbt.gg/shawnp0wers and sign up for a trial!
The 4 Part Series on Ansible includes:
Part 1 – DevOps for the Non-Dev
Part 2 – Making Things Happen
Part 3 – Playbooks
Part 4 – Putting it All Together