Data Migration FAQ | Frequently Asked Questions

What is a data migration?

At its core, a data migration is the process of either:

  1. converting data from one or more formats, into a different format or system
  2. moving data from one or more storage locations, into a different location, or
  3. a combination of both; that is, converting data from one or more formats into another, while also changing the storage location.

What is the process to migrate data?

If you perform a search on google, or research this question, you'll most likely find that the industry standard process is to use ETL: the extract, transform and load method for migrating data. We find this method to be flawed in that it is missing a key pre-requisite step. Feel free to check out article on Misunderstanding ETL to learn more.

Our general approach and methodology for any project we work on, including data migration projects, is:

  1. Understanding the problem / goal
  2. Analysis
  3. Requirements
  4. Development
  5. Testing
  6. Feedback Loop
  7. Deployment/Execution
  8. Support

Our approach specific to data migrations is to use a new and improved process, called A-ETL, where the first step is analysis. We've put together a diagram showing the general process:

New & Improved ETL Diagram

2018 - New and Improved A ETL Diagram

Do you offer free tools for the public?

We sure do. The following is a list of some of the tools we offer for the public and what they do:

  1. MySQL Data Encrypter
  2. This tool allows you to encrypt your data. Simply drag'n'drop your database table (exported to a CSV file) onto the tool's drop zone, and we will automatically create a database table, ingest your data, encrypt it, and present an interactive database table explorer that allows you to explore and query the data. We also provide the special queries on how to decrypt the data.
  3. Batch CSV To Data
  4. This tool allows you to batch upload CSV files, and have them all automatically ingested into a MySQL database. This allows you to write queries against your data, rather than trying to search CSV files one by one, and is far better than trying to use a linux grep or other file content search method.
  5. Batch Image Converter
  6. This tool allows you to drag'n'drop a whole folder or directory of image files, and convert them into a different format. It also has options to resize your output images to any size, including thumbnails.
  7. Nested File and Folder Analysis Tool
  8. This is a great tool for profiling a data set. If you have a directory of files, with sub-directories and more files etc, and want to profile it (eg, determine uniqueness based on the file's checksum, or query for filenames, or extensions, etc), this is a great tool.

Note that all of the tools include additional information, as well as a technical how-to guide on how you can develop a similar tool for yourself.

How do I use the free tools that are available?

Simply visit our All Migration Tools page, or simply click on the individual tool links in the question up above.

I have an idea for a tool that the public can benefit from - will you develop it?

We are certainly open to it. Send us an email and let us know what kind of tool you think would be a good addition to our site, and internally we will discuss the possibility of creating it. Note that any requests for us to develop new tools should be for a tool that benefits other technical resources within the general public, and it should be a tool that isn't readily available on other websites.

Can I use diagrams on this site on my own website?

Yes - if you find any diagrams on our website that you'd like to use on your website, feel free to use it. If you do use one of our diagrams, you must use the diagram via a src link to our website, for example: <img src="[some diagram].png" />

Do you offer services other than data migration services?

Yes, in addition to data migrations, we do custom application and/or web development, proof of concept software development, automation, batch processing, custom reports, analysis and more.

What is the price for your services?

For record or file conversions, we have a fixed price per record or file. For data migrations, we generally offer a fixed bid price; we have found that many business can be adverse to a Time and Materials arrangement, due to past experience with other service providers where the project exceeded both time and cost. Instead, we offer a fixed price bid, so that you'll know before engaging with us exactly what the cost will be. However, if there isn't enough information on the project scope prior to engaging, then we may present a fixed bid for the initial development, with a T&M type of engagement for future updates and maintenance.

Our company needs a custom program to automate a task, or to migrate data - can you develop it?

That's our wheelhouse - we recommend that you contact us for a Free Consultation in order for us to learn more about your project and business objectives, and if you're a good fit to work with us, we can begin the process of working towards a solution.