A Deeper Look into 4D Data Encryption

Encryption is an essential part of your 4D data protection and your organization’s strategy, especially with GDPR, HIPPA, Sarbanes-Oxley, and PCI Data Security Standards. 4D provides a powerful built-in encryption solution to help you comply with different privacy laws and regulations while keeping your and your customer’s data safe and sound from intruders. 

This document provides a deeper look at how 4D encryption works so you can use the feature to its fullest potential. 

 Introduction to 4D data encryption

4D offers several great avenues for keeping your data secured. One of these avenues is to secure the data itself using dedicated built-in encryption. Just as we lock and restrict access to our homes in the physical world, we rely on encryption to keep intruders away from our data. Data encryption is the mechanism of transforming readable data into an encoded form so that only the person with the data encryption key can read and update it. So, if the database is stolen, the thief would have no way of opening and reading or extracting your data without also having the encryption key.

Let’s take an example of a 4D application; when opening a .4DD file in an external editor, it was previously possible to guess some of the data content.

Run the following code:

var$e;$status:$OBJECT
$e:=ds.MyTable.new()
$e.Field_Alpha:="mydataaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
$status:=$e.save()

Now edit your .4DD file. You can recognize your data:

But that was then…

Now with 4D’s encryption, once data is encrypted, it becomes completely unreadable:

 

Why should you consider 4D data encryption?

4D’s built-in encryption was designed with performance, robustness, and simplicity in mind.

Performance

Compared to many other products, the 4D solution delivers better performance since, instead of encrypting the entire database, 4D offers table-level encryption, which results in a low to no impact on performance. That’s why it’s recommended to group sensitive information into targeted tables.

Robustness

In addition, 4D encryption uses a 256-bit key instead of the 128-bit, which is used by products similar to 4D. That 256-bit key is much more robust from a security standpoint (as explained in this section). Stronger keys are harder to break for attackers. As a result, a brute force attack is much less likely to succeed against 256-bit encryption.

Simplicity

All this power was built beneath a veneer of simplicity; the 4D encryption process is made as light and straightforward as possible for 4D developers and administrators by:

  • Integrating the encryption process into the MSC via a user-friendly and intuitive interface that you are already familiar with,
  • the availability of a set of 4D commands that automates the encryption process.

 

How Does 4D data Encryption work?

    the algorithm and technology behind

Encryption converts plain text into an unreadable message called ciphertext. This process requires two elements: an encryption algorithm and a key. Decryption is the reverse process of converting ciphertext to plaintext and also requires a decryption algorithm and a key that only authorized users must have.

Generally, the encryption and decryption algorithms are the same. The key is used when encrypting the plaintext and when decrypting the ciphertext. The choice of the key is essential because the algorithm’s security depends directly on it.

Encryption algorithms can be divided into two categories:

  • Category 1: where a symmetric encryption key is utilized. In that case, the same key is used for encryption and decryption,
  • Category 2: here, an asymmetric encryption key is utilized. In this case, two different keys are used for encryption and decryption, one private and one public.

 

4D data encryption is based on a symmetric algorithm: The Advanced Encryption Standard (AES) algorithm, which is robust, easily optimizable, and has already been proven by the community since it’s open-source. Encrypting and decrypting actions require a passphrase. Like a password, a passphrase is composed of a sentence or a combination of words. It’s used to generate a binary encryption key to encrypt data with the AES algorithm. More technically, a 256-bit SHA is generated from the passphrase and creates the AES binary encryption key. Therefore, the passphrase or the binary encryption key is mandatory to work with an encrypted data file.

 

   

    the Encryption process

First things first, 4D encryption impacts the following files:

  • The .4DD: the file that contains the data,
  • The .4DIndx: indexes file,
  • The .journal: log of events occurring on the data. The log traces all the actions that have been made on the database. It can therefore contain data (in case of insertion or update).

 

As for the encryption process, it’s based upon a simple, two-step scenario:

Step 1: Selecting tables to encrypt

  • The developer selects tables to be encrypted at the database structure level.
  • For performance reasons, only tables containing sensitive data need to be selected. 
  • No encryption is done at this step; users can normally work without data encryption.


Step 2: Encrypting selected data

  • The developer enables encryption using the Maintenance and security center (MSC) or the 4D language via dedicated commands.
  • Encryption is then propagated to all pre-selected tables.  

 

    the decryption process

As for the decryption process, when a data file is encrypted, the developer can work transparently with the flexible key management system 4D provides (4D KeyChain). Once a data file is opened, 4D will automatically check the 4D keyChain for a valid key. 

If there is no valid key, the search is performed for a possible device (USB keys or disks). If a valid key is found, it will be stored in memory in the keyChain, and access is given to the encrypted file. Suppose no valid key is located in the keyChain or in a device. In that case, there will be no access to the encrypted tables unless the developer provides an interface to type the passphrase to provide it to the database via the provideDataKey() command.

The only access to non-encrypted tables is possible if no valid key is provided.

This diagram summarizes the process: 

blank

    4D keyChain

4D keyChain is a flexible key management system provided by 4D. It’s a space in memory where the encryption key is stored. As long as 4D is not closed, it’s possible to open and close data files without communicating the key again.

The .4DkeyChain files can contain several keys. The key will be automatically detected when the device is connected when opening the database. As soon as a valid key is detected on a device, it will be saved in the keychain so that the device can be disconnected while using the database. This also allows working with several data files without entering their passphrase. The command, Discover data key(), also allows scanning a device to find a valid key if the device has been plugged in after opening the data file.

    A note about journal and backup

A journal is associated with each data file. This log traces all the actions carried out on the database. So, when you want to encrypt a database, you have to associate a new log with the same encryption key as the data. The encryption involves the data, the journal, and the index files. Keep in mind that the backup is also impacted by encryption. The resulting backup file will also be encrypted if a backup is made on an encrypted database. Of course, the encryption key will be required if the backup file is restored.

    Client/Server encryption 

In client/server mode, you have to provide the key to the database opened on the 4D Server. This makes encrypted data available to all clientsr. To do this, you can either:

  • open the database on the 4D Server with the encryption key on a USB key or at the root of the disk
  • provide the key to the database by executing a method on the server that does:
    • a Discover data key() (but the encryption key must be on the USB key or on the disk)
    • ds.provideDataKey()

The second method can be run from a remote 4D.

 

How to use 4D encryption?

Encrypting your 4D data is straightforward: First, choose the data to encrypt, then encrypt it via the MSC or dedicated 4D commands.

The new Encryptable attribute is available for tables in Design Mode. Encryption is not executed here; this sets the encryption status by designating that it can be encrypted. 

blank

     Encrypt data using the user interface:

Once you’re ready to encrypt your data, the easiest way is via the Encrypt page in the MSC. To encrypt your data for the first time, click on the Encrypt data button. 

blank

 

You’ll then be asked to choose a passphrase. 

blank

The MSC Encrypt page provides all of the necessary features to monitor your data encryption. You can use this page to encrypt, decrypt, re-encrypt the data file and change your passphrase.

     Encrypt data using 4D commands:

Using dedicated 4D commands is an additional way to work with encrypted data files. These commands are designed to support most encryption requirements and enable you to deliver an encrypted solution to your customers. They are also beneficial when the MSC is not accessible, such as when the database is launched as a Windows service.

4D puts  seven commands at your disposal to get up and running:

  • Encrypt data file: As the name suggests, this commands enables you to encrypt or re-encrypt the data file designated in the first parameter. 
  • provideDataKey: In addition to 4D’s automatic data encryption key detection, this command is available in the ds object to provide the data encryption key to an opened data file. This is useful when building your own user interface to control access to your encrypted data. While ds is an ORDA feature, this command is valid for ORDA and classic 4D code.
  • encryptionStatus: This command is also available in the ds object. It checks whether or not the opened data file is encrypted and if a valid data encryption key has been provided.
  • Data file encryption status: It returns the encryption status of a specified, unopened data file. This allows you to verify the encryption status of a data file before opening it.
  • New data key: Generates a binary data encryption key from a given passphrase. Since 4D automatically searches for a valid data encryption key on connected devices when a data file is opened, this command allows you to generate your own binary data encryption key, which you can then write to a file.
  • Discover data key: If you forget to connect the device containing your data encryption key before opening a data file, you can use this command to provide the key.
  • Register data key: To avoid providing the data encryption key each time you open a data file, you can use this command to add it to the 4D keychain.

 

Here is a database example to see these commands in action. If you are using 4D v19 and above, use this one as it supports modern code.

 

Where to go from here?

Many resources are at your disposal for a deep dive into this topic.

Blog posts:

 

Videos:

 

Official documentation:

 

Community:

 

Note: Please feel free to share your feedback with us on the 4D forum and your experience using the encryption feature. We are looking forward to reading you ? 

FAQ

What happens if I lose my decryption key?

If you lose the decryption key, access to that data is permanently lost.

Is performance affected by encryption?

Compared to other products, the 4D solution delivers better performance since, instead of encrypting the entire database, 4D offers table-level encryption, which results in a low to no impact on performance. That’s why it’s recommended to group sensitive information into targeted tables. Also, note that decrypting the records is done between the disk and the cache. Once in the cache, the records are not encrypted anymore. The same is true for the index pages because their content is also encrypted on the disk.

How to restore a backup that has been encrypted with an old key?

If you wish to restore a backup that has been encrypted with an old passphrase, you must remember this passphrase to access the restored encrypted data.