Accessibility Tools

Skip to main content

Secure Data Deletion

Data Deletion and Erasure

When something is deleted, most people expect that it is no longer accessible to them or anyone else. Unfortunately, this is far from the truth. Deleting data does not mean that it is no longer accessible.

This is edited content from Sarb Sembhi and co-host Nick Ioannou. Also in video and podcast media.

 Glossary Terms in this Blog Article - hover to view, click for full glossary.

Introduction

When data is not effectively deleted not only is it possible for others to access it at a later time, but it is also possible that it could result in non-compliance with data protection policy. Today we are looking at why deleting data doesn’t mean that it is not accessible, how to ensure effective data deletion, and what happens when that doesn't happen.

Issues with data deletion

When we delete, amend or alter any data, certain elements of the original file may still be accessible. One of the best-known examples of this happened during UK prime minister Tony Blair's era, when attackers managed to access - and leak - a deleted intelligence file about Iraq. Although the file had been deleted, the unauthorised user was able to access meta data that contained the name of the document's author, along with other information.

Large and small organisations alike have been caught out and embarrassed by deleted information becoming public, often after donating or selling old technology such as portable media (memory sticks and portable drives) or computers (laptops, PCs and Macs) to charity. It's only later the organisation realises that the technology fell into the hands of someone who knew how to access that data and make use of it.

There have been many examples of academic institutions who instruct their students to purchase second-hand technology to research and explore what data they could find on these devices.

These examples illustrate that those companies selling used technology don't understand the issue, but also that even when studies have explained how easy it is to extract data, such studies have had no impact on people’s behaviour.

"It is easy for security researchers to illustrate how trivial it is for them to not only access the data but also to use the confidential data for things that the original owner never intended."

It is easy for security researchers to illustrate how simple it is for them to not only access apparently 'deleted' data but also to use the confidential data for things that the original owner never intended. However, not all of the donated or sold media or devices end up in the hands of researchers. Many end up in the hands of people who are only looking for cheap things in the hope of retrieving compromising data that could be sold on for more than they paid for the media or device.

Since second-hand technology is so cheap, many criminals have used such approaches as side-line organisations to extract and extort the seller on the compromised data found.

The question that we often hear is, “If it is so easy, why doesn’t someone do something about it? To answer this question, we need to understand how data is stored and what happens when it is deleted by the user.

What happens when data is deleted?

The easiest way to understand what happens when data is deleted is to understand what happens when data is created. The process is similar on all technologies, although there are variations on different operating systems, different versions of operating systems as well as different types of media, and also encrypted drives.

When a user creates any new data, it will be stored in a file, and that file will use a standard that defines at the start of the file how it should be read. This includes information about the logged in user who created the file, the time it was created, the language, and such things. This type of data is meta-data, and needs to be understood by the application to read the file. Incidentally, it was this information that the UK PM’s team had missed amending when they adjusted the rest of the document. And it is also this information that some students who have been caught plagiarising assignments forgot about when they handed in documents as their own original work.

However, with regards to how the system knows that the file exists on the media (be it a traditional hard disk, an SSD, a memory stick, etc.) usually the system will use something called a look-up table to create a record of that file. This states where on the hard disk the file starts and where it finishes. In effect this is an indexing of every file that is created. Not only does that information get recorded, but over time as other files may get created immediately after the previous one, and new data on earlier files will be added to the end, which means that the file will be physically split on the media. So, the look-up table records each new addition to the original file and all the new fragments of data related to that file.

When a user deletes a file, what happens is that the entry of that file listingand all its different fragmentsis deleted.

In reality, this means that although the file still exists, only the entry and use of the space on the disk has been deleted from the look-up table.The effect of this is that the system is still able to undelete the file, which is why in most operating systems like Windows, a user is able to access their Recycling Bin and restore a file. It is because it was never actually deleted in the first place - only the look-up entry was deleted.

Even when a user empties their ‘Recycling Bin’ what happens is that the operating system is no longer able to access the look-up table and where the data existed. What this means is that in most cases it is possible to recover data using the right tools. Some are free, and some are for professional computer forensic staff and cost a lot of money.

Deletion and erasure

Professionals use the term data erasure to differentiate between data that is merely deleted, and data that has been completely removed. Data deletion is exactly what we all do when we delete a file, but data erasure is the term used for completely removing all possible remnants of the existence of previously stored data. It is this process that should be used when cleaning any devices for resale or when donating it to charity.

"Different people use different phrases to mean data erasure be it secure data deletion or data wiping, data destruction or anything else."

Different people use different phrases to refer to data erasure - it might also be called secure data deletion, data wiping, data destruction or something else. Each of these terms are intended to indicate that it is something that is final, in that the data should not and will not be recoverable. The question that most organisations often want to know is how do they securely delete confidential (or any) data so that it cannot be recovered by anyone else?

One of the most quoted pieces of research on this topic is a US military study that stated that the way to ensure that a file or data is no-longer extractable at all, is to delete and overwrite the same physical space of the file on the media at least32 times. This means that a user or a tool must delete and overwrite the data multiple times so that it is no longer decipherable.Such tools have existed for several years and available at no cost, and some operating systems as well as anti-malware tools include utilities that make this task easier.

Secure erasure is the process of ensuring that a file, folder or collection of files or folders can no longer be recovered by existing tools with any level of confidence.

Further, it is not only these tools that are able to achieve this end result, the other way to achieve the same result is to use file or disk encryption and then not provide a decryption key.

Security professionals always recommend using the most appropriate encryption for the appropriate job. Basically, if the right tools are used with the appropriate standards for encryption, it provides the right assurance that the information cannot be accessed without the appropriate credentials. However, once digital files are encrypted (with the right encryption) since the data cannot be accessed, the effect is nearly as good as erasing them. This assertion is based on the assumption that the encryption standards used will never be broken.

"Some organisations encrypt all their devices as well as the data on them."

 

Some organisations encrypt all their devices as well as the data on them. The second level of encryption provides additional assurance that even if the data on  device is not securely deleted, there is less of a concern, in cases of lost devices.

Any organisation using the current recommended standards for encryption will also be able to have a level of confidence on their data being in-accessible by others when they dispose of digital media, as long as the encryption key standards were followed – if not, it would be the same as having weak passwords to access the data. This is sometime seen on the television when the techie person will be told to break the encryption – so next time you see this, you will know that they are doing.

Why is data erasure important?

Data protection legislation mandates that every organisation is responsible for the personal data it processes, and what happens to the data once it has finished with it.

Further, a organisation doesn’t want to lose its intellectual property, trade secrets or any information that may damage its reputation. Not using effective data erasing practices is a form of unintentional data leakage which could have devastating results if not, hefty fines as well.

Data erasure is and should be considered usual good practice when disposing of any media with data on it. There have been many examples of investigators and researchers actively looking to identify where all old media belonging to specific companies ended up, with the aim of being able to recover the data on the media.

Although in most cases data erasure is now simpler than it has been for many years, it is not so simple, because erasure depends on several factors mentioned earlier, including operating system, version of the operating system, type of media, formatting, etc. But if undertaken effectively, it does produce the intended outcome of ensuring inaccessibility of the data. At this point we often get asked, “Does that mean we can never recover the data?"

Actually, that is the whole point, to stop anyone even the organisation or people who created the data from recovering it- ever. If it needs to be recovered, then it just shouldn’t be deleted in the first place. There are no grey areas in this, either data needs to be kept, in which case it is secured, or it doesn’t, in which case it should be securely erased.

What is a data retention, deletion and destruction policy?

Many organisations have data classification policies and data retention, deletion and destruction policies to ensure they deal with data appropriately at the right time, using a consistent approach.

The time this becomes relevant varies depending on many factors, but when the organisation starts to create or hold a lot of personal or confidential data, that is usually the best time. For many small organisations it could be when it is a single person organisation, or as late as when it starts to employ more than just a hand full of people.

As part of complying with data protection legislation, many organisations will have a policy for how long they keep data, how they delete it, and how they destroy both physical and digital data. For some small organisations, it may be useful to put these policy statements into their data protection policy, but for others it is often more useful to keep it separate, as the types of data they process is more complex.

The complexity may arise out of the data used being of different types, format, where it is held, etc.which will determine how that data needs to be dealt with.

Personally Identifiable Information will need to be dealt with differently than Sensitive Data, which will be dealt with differently if it is stored in one country or another country. Or, if the data is held in a database, or a physical copy. 

What are the issues with some file formats?

With physical paper-based data, most organisations are already familiar with using shredders, and use specialist companies to remove and burn the shredded paper. This is not feasible or practical for all file formats.

"There have been numerous press stories where a disposed computer held unencrypted database information."

Data in databases can be a particular case in point, since there have been numerous press stories where a disposed computer held unencrypted database information. Several years ago, using encryption in a database would have added additional overheads on the server that would slow down the processing, and often organisations were reluctant to encrypt database data. Now, with cloud and other technologies, this is not as big an issue. Organisations can encrypt new organisation data should they wish, but the challenge is that there is a lot of data around which is legacy data sitting on legacy applications and databases. This data may have been kept for legal reason and cannot be deleted, but also that the cost of encryption would be too high.

In the cases of database information, where it was not encrypted deleting individual records may be unfeasible, organisations will need to delete the whole file using secure deletion tools.

Different approaches to eliminating access

The technology behind each device and the storage media it uses is different, as are the many other factors mentioned above. So, for example, the way that the two dominant mobile platforms store data is different when it comes to their system data, however, how they record existing file formats they stick to the prevailing standards for those formats. The challenge for a organisation trying to dispose of the various types of technology which may hold its data will require a different approach.

Some of the tools available to anyone who wants to erase data from different types of devices may be able to use the following applications.

  • To ensure that Windows operating system devices are still useable after the process of erasure, the following free tools will erase data in accordance with the standards based recommended guidelines: Eraser, Secure Eraser, File Shredder, also some anti-malware suites also include such tools.
  • To ensure that the media is no longer accessible to anyone at all, it is possible to utilise a shredding service, which will reduce the disk media to a mass of shredded shavings which no amount of piecing together can make useable.
  • To ensure that the media is possible to use, but the data is not, it is possible to use disk encryption, where in some cases professionals may encrypt the whole media several times and then erase it before installing a new operating system on it.
  • Where the device has a Solid-StateDrive (SSD) the easiest way to permanently erase data from these is to use the manufacturers own tools, although there are plenty of commercial tools available.
  • To erase data from mobile phones and tablets:
    • For Android devices, we know that a factory reset is not up to the job. However, for more recent devices if the device has been encrypted first before use, a factory reset will delete the encryption key and render the data inaccessible. This is good enough for most devices currently in use
    • For iOS devices which are encrypted by default where the encryption keys are handled by Apple. When a user deletes the data and prepares to remove their profile data and apps it will become inaccessible to any future user of the device
    • For portable media likeFlash, CDs, DVDs, Tapes, Cartridges, RAID devices, etc. If a organisation is still using these, it is because it has been around a while, and the data on such media is very old, one of the easiest ways to erase data from this media is to employ the services of a commercial service to destroy the media
  • To erase data from cloud services, it is best to use the same approach as that mentioned above. Encrypt all data going onto the cloud platform so that if it is deleted since you as the only user has the encryption key will be able to undelete it (if it is possible), or it is inaccessible to anyone else
  • For physical paper documents many organisations may just use a shredder, however, it is important to make sure that a cross-shredder is used, and that the shredded remains are disposed of correctly. Many larger organisations outsource this to providers who incinerate the shredded paper.

Using commercial services

As with anything related to security, it is important that when considering using a commercial service a specification of what is required is documented and then confirmed that a prospective supplier is able to meet it. Amongst one of the requirements, it is important to understand the extent and limitations of the service. In some cases, it may be important to ensure that they provider is able to provide a Certification for the service. In other cases, it may be important to ensure that the provide holds current security certifications for the line of organisation they are in.

Conclusion

Life would be very simple if deleting data from a device really did delete it, however, it may then be unrecoverable.

Data Erasure using the appropriate tools is what organisations should be doing, if the intention is to ensure that the data becomes inaccessible to anyone, in cases where a device has come to the end of its life in that organisation.

Where appropriate organisations should have “Data Retention, Deletion and Destruction policies in place, so that employees are aware of what is expected of them, and to ensure there are no ambiguities by which the organisation may be caught out by data protection legislation.

 

Default sample Threat Map infographic


Infographic images are copyright of Virtually Informed, and available to registered users for download during the publication week of the blog article together with other downloadable resources, including: all related infographics on this page, example policy templates, posters, screen savers and much more. 


Images from https://www.pixabay.com.