SSH Keys in macOS

For security, a number of services may use SSH keys for authentication rather than a standard username/password.  Some services include:

·       SSH

·       GitHub

·       GitLab

Keys add a level of security that standard username/passwords don’t provide.  When a key is generated, two files are created, a private key, and a public key.  Those two files are mathematically/cryptographically tied together in a way where a separate key would not be able to unlock it, hence the term ‘keypair’.

Prior to being able to authenticate, you need to give the public key to what you are intending to authenticate to.  Therefore, when you do authenticate, your private key (that is never given out, EVER), is used to encrypt/decrypt data that only the public key can subsequently encrypt/decrypt.  Only that unique keypair will be able to work together.

Creating a new KeyPair

In macOS, by default, SSH keys are stored per user in their hidden folder ~/.ssh.  To create a new keypair:

1. Open Terminal.
2. Run the following command:

ssh-keygen –t rsa

Once executed, you will be prompted with a number of questions:

-Where to save the key.  Press Enter here, as its default (~/.ssh/id_rsa) is sufficient.

-Enter Passphrase.  It is a good idea to pick a password to use instead of leaving empty.  The security of a key, no matter how encrypted, is still dependent on the fact that it is not visible to anyone else.  Should your private key fall into someone else’s hands, they will potentially have access to all services that use said key, UNLESS you have password protected said key.  The caveat is that you will need to type in the passphrase each time you use the key.

Once generated, both files (id_rsa and id_rsa.pub) will exist in your ~/.ssh directory.  In order to properly use the keypair, you will need to distribute the id_rsa.pub file to each server you intend to authenticate to.  This will vary depending on the server.

Changing the Keypair Passphrase

To change the passphrase of an existing keypair:

1. Open Terminal
2. Run the following command:

ssh-keygen –p –f ~/.ssh/id_rsa

…then supply your old and new passphrase (twice) at the prompts.

Testing Large Scale Storage

I’ve recently had the pleasure of testing a few large scale storage systems and my co-worker and I needed to come up with a number of tests to determine their overall performance when it comes to Macs.  These tests I’ve used in the past give a very good overall idea of any platform you are testing:

(Hint) Use a high performing system with an SSD in order to avoid local performance bottlenecks.

#1 Test Read/Write Speed to the Volume
    Determine the various file share protocols the platform can host, then attempt read/write tests across each protocol.  Protocols may include:
    -AFP
    -SMB
    -SMB with Packet Signing Disabled (Packet Signing is great for security, but does degrade SMB performance.  See this Apple Support Article: https://support.apple.com/en-us/HT205926)
    -CIFS
    -NFS

    I use the BlackMagic Disk Speed Test available from the Mac App Store: https://itunes.apple.com/us/app/blackmagic-disk-speed-test/id425264550?mt=12
…then document the results for each protocol in turn.

#2 RSYNC Speed Tests.
    We use this test in order to determine the real world performance of transferring data, between individual LARGE files or thousands of SMALL files, as write performance degrades when having to transfer many small files.
    -First, from your test system, create a folder, ‘cd’ to said folder, then generate 10,000 1MB files via the command :

for i in {1..10000}; do dd if=/dev/urandom bs=10000 count=100 of=file$i; done

-Secondly, create a separate folder, ‘cd’ to said folder, then create one large 10Gb file via:

mkfile -n 10g /Folder_Path/10gFile

-Once these files are created, use RSYNC to transfer the data once for the small files, and once for the large file to the destination:

rsync -a —progress —stats —human-readable [source] [dest]

Once transferred, document the results of the RSYNC to get real world speeds.  You can try this off of the various protocols as well if need be.

#3 Finder Display Performance
    What most people don’t know is that before you actually see items in a folder, the Mac filesystem first iterates every item in it so it knows WHAT to display, as well as HOW.  Finder will seem like its is unresponsive when trying to iterate those items, and you can test the Platform to see how fast Finder can access it to iterate all of the items.
    -Simply get a timer on your phone, navigate to the parent folder of the Folder that has the thousands of small files.
    -Change your Finder view to Icon view.
    -When ready, open the thousands of files folder while simultaneously starting the timer.
    -Eyeball when Finder actually displays the items, and stop the timer.  Record the time it took.
    -Repeat the process for the List view, as well as the Column View, as the views will respond differently.

#4 Trailing Whitespace Management
    One of the biggest pain in my ass is the difference between OS X/macOS and Windows in how they deal with spaces at the end of filenames.  A lot of times, file transfer protocols deal with unique characters differently, which can really cause problems depending on which OS access the file.  
    -Create a new text file with a whitespace at the end in windows on the file share (SMB) called “Testfilewithwhitespace .txt”, then mount the share via AFP on the Mac, and rename the file to “Testfilewithwhitespace ”.  A lot of times you will see that Windows completely renames the file to garbled text due to an old feature called Name Mangling.  http://www.oreilly.com/openbook/samba/book/ch05_04.html.
    -You will want to test yourself to see now the platform filesystem deals with trailing whitespaces to learn how it will handle them.

#5 Hidden Files
    OS X/macOS can hide files if they are named with a . at the beginning of the filename, eg “.HiddenFile.txt”. You will want to determine how the Platform Filesystem tracks these files.

#6 Authentication Support
    Larger organizations are most likely using a Directory system of some type, either Active Directory, Open Directory, OpenLDAP or others.  You will want to determine whether the platform you are testing supports your Directory of choice.

#7 Permissioning Tests
    Once you know the support of Directory Authentication, look and see how the Platform deals with permissioning.  Check whether it supports POSIX and/or ACL’s as well as HOW to adjust them.  Test your file share protocols to make sure that permissions are properly followed by the test system.

#8 Microsoft Office Files
    Office is notorious for file saving errors related to creating lock files when files are edited directly on the share.  Officially, Microsoft says to copy the file locally, edit it, then copy it back to the file share when done, but you can test this by looking for the ~… temporary lock file that gets created when in the middle of being edited.  Once done editing, the lock file is removed, but there can be errors where the lock file stays after the fact.  Look on your platform to determine how best to remove those files after the fact.  Separately, look for whether the /.TemporaryItems folder gets created at the root of the share.  See this MS forum thread for more info https://social.technet.microsoft.com/Forums/en-US/f0ab739a-a649-45df-b14c-13aacf5b9192/issues-saving-office-2016-for-mac-files-to-a-shared-volume?forum=Office2016forMac.

#9 Test Your Own Business Workflows
    Where I work, Adobe files are prevalent, so it behooves us to test various Adobe related workflows like the time it takes to transfer hundreds of JPEG’s, or the time it takes to open multi-gig banner .PSB files directly from the share.
    You will know what custom business workflows you might run into, and it would help to test those.

#10 Ultimately, record your own Thoughts and Opinions
    Sure, we are dealing with lots of hard data, but it does also help to record your own opinions of the system, so others can get info.  How does this compare to previous platforms you’ve already tested?  How do you feel when working with its admin console?  Are you comfortable with it, or do you feel like there will be a steep learning curve when it comes to its administration? …etc.

Hopefully these bulletpoints help if/when you ever need them…

Locating a Rogue DHCP Server

One of the more difficult situations I've been a part of the past few years is trying to figure out why a computer on a network starts randomly using a different set of network settings than what you know they are supposed to have.  This usually results in the computer not being able to connect to other devices or access the internet itself.

In rare circumstances, the issue may be due to a rogue DHCP Server on your network.  Remember that DHCP is the protocol used to automatically configure a device so it can talk to other devices that fall under the same network settings.  At a simplified level, when a computer connects to a network (and doesn't already have network settings configured) it will 'yell' out on the network asking who can give it the network settings needed.  A DHCP Server on said network will reply back with the required settings, at which point the computer should be able to properly connect.

However, what happens when a network has multiple DHCP Servers?  How does the computer determine which network settings to use if multiple DHCP Servers respond?  Simple: it will use the settings from the first one to respond.  There is no easy way to tell computers to only trust certain DHCP Servers, without breaking the whole point of DHCP in the first place.

I have not had good luck at finding a tried and true quick fix, but the following steps have been rather successful for me.  It uses Wireshark to look at raw network traffic, which can be REALLY daunting to look at, and I personally am VERY much a noob when it comes to Wireshark, but it is quite powerful.

1- Before anything else, install Wireshark on your computer from https://www.wireshark.org.  

2- Connect whichever network interface is needed to the offending network (Ethernet, Wifi, etc).

3- Once connected, open Wireshark, locate the interface you are using, then double click it to start a new 'live capture session'.  This will trigger Wireshark to start listening to ALL traffic on the network and will display each packet it can get its hands on.

4- Leave Wireshark running and renew the DHCP Lease on your computer.

5- Once the release has completed and you have a new (or same) IP address, go back to Wireshark and stop the capture.

6- Leave the capture open and type in 'bootp' in the Display Filter field, then hit Return.  This will automatically filter the entire capture for packets related to DHCP.

7- What you now see is each packet the capture found that is related to DHCP listed in the top field.  Highlighting a packet will show you the details of the packet in the middle field, and the raw packet data in the bottom field.

8- You need to look for a packet with a source of 0.0.0.0, a destination of 255.255.255.255, and a Info field showing 'DHCP Request'.  This packet should be your computer yelling out to the network asking for network settings.  You can confirm it is your computer by comparing your network interface MAC address with what is listed in the packet details.

9- Now that you have found your DHCP Request, look for packets afterwards that are 'DHCP ACK' packets.  These are packets from the DHCP Servers ACKnowledging your request and replying with the actual network settings.  If you feel like it, you can peruse the details of the packet in the middle field to see what the packet contains.

10- If you have a rogue DHCP Server on your network, you should be seeing MULTIPLE 'DHCP ACK' responses to your computer coming from different Sources.  One of these sources is your rogue which you will need to track down.

---

Now that you know the IP address of your rogue DHCP Server, the only other issue is physically locating the computer which might be tougher in your environment, which is not NEARLY as easy.  If you have access to the switching hardware or a network administrator, you can use the ARP Table(s) on your switches to locate the specific ethernet port a MAC address is sitting on, and trace back the ethernet cable to the offender.  Talk to your network admin about accessing those.

Were you affected by the CloudFlare HTTPS Traffic Leak? You should change your password...

CloudFlare is a rather well known web services and security company that MANY public companies use for hosting and website reliability.  Unfortunately, it came to light a day or two ago that a bug existed in their system that allowed private passwords, messages, API keys and other sensitive data to be leaked to random requesters.

This means that any of that data was cached by search engines and other random systems.  It sounds scary, but the scope of the leak is immense.  Over 4.2 million domains were affected, and so it is probably a good idea to change your passwords, but how do you know which accounts were compromised?

A user on GitHub put together an ongoing list of domains that use CloudFlare, which is accessible here: https://github.com/pirate/sites-using-cloudflare

To check, download the full list, unzip it, then open a command line prompt and run

grep -f domaintocheck.com sorted_unique_cf.txt

...to see if the domaintocheck.com is present.

If you want to check a bunch of domains at once, create a .txt file with all potential domains one per line, then run the following:

grep -xF -f [.txt file] sorted_unique_cf.txt

...which will output each domain affected.

Verifying the contents of a folder were copied correctly

One of the commands built into OS X is md5, a way to create a simple 'fingerprint' of the contents of the file.  The idea is you can verify whether a copy of a file is the same, by making sure the two md5 'fingerprints' are the same.  Execution of the command is simple:

md5 [filepath]

...which will then output the resulting fingerprint, or hash.

However, how do you do this for verifying an entire folder?  Use the following, which takes each file inside a directory, creates a hash for each, then re-hashes all of them together again to get a final fingerprint, which you can verify on the copy.

find DIRECTORY -type f -exec md5 {} + | awk '{print $1}' | sort | md5

Im still somewhat of a newbie when it comes to these sorts of things, but if others have recommendations when it comes to this sort of thing, by all means let me know.

Creating a PKG from a drag-and-drop DMG

Quick Terminal command for those one off applications that need to be deployed via a .pkg installer:

sudo pkgbuild --install-location [Destination_Path] --component [Actual_Application_Path] [Save_Location].pkg

For example, if you have EazyDraw 8 currently installed in your /Applications folder: 

sudo pkgbuild --install-location /Applications/ --component /Applications/EazyDraw\ 8.app/ ~/Desktop/EasyDraw8.pkg