For many years, 4D has allowed you to develop binary databases as part of a team with 4D Server. This way of developing is straightforward, but many developers asked us to be more efficient in source code management for better traceability. We heard you, and we developed project mode to fit this need.
This guide will help you understand how Version Control Systems work and how they could benefit you. It’s divided into four sections:
Version Control Systems (VCSs) are designed to manage source code based on text files. Many version control systems exist, and they all answer important needs in professional development environments:
- Traceability: who did what, when, and why
- Security: easy backup and rollbacks
- Shareability: working on the same resources without locking them
With binary databases, 4D Server fulfilled both roles of data server and source code server. Since the source code was integrated into a monolithic binary file, no VCS could manage source code history.
Following that, 4D imagined an architecture that could benefit from VCSs and make development easier and safer for teams: Project mode was born. This mode transforms the monolithic binary file into several text files, each one representing an element of the original database (methods, forms, resources, settings, …). A VCS can easily manage 4D projects and take charge of the source code management part of 4D Server.
We won’t base this guide on a particular VCS tool, because the possibilities offered are similar from one to the next. The differences between those systems are mainly their architecture, which we’ll explain in this overview.
Keep in mind that the way you develop with projects in a team doesn’t impact the deployment and execution of compiled applications with 4D Server or merged 4D Servers.
Finally, this guide is not only for team developers: the benefits of using VCSs in team development are also relevant for single developers!
What is A Version Control System?
This chapter explains the concept behind version control systems. There are many version control systems (Git, Subversion, Mercurial, and so on). This is just an overview, you can find more details on these technologies on several websites.
All systems allow defining a reference repository (the source code library) that can be located on a computer, a local network server, a private cloud server, or a public cloud server, and many of them allow hosting of the reference repository in a private or public cloud service.
As we’ve seen in the introduction, the main goal of all version control systems is to answer important needs in professional application development:
- Traceability: all systems store source code modifications and track them
- Security: all systems store backups of sets of modifications defined by the developer and roll back the security
- Shareability: all systems allow developers to work locally on their own computers and manage conflicts when many developers have touched a resource
Version control systems use two major architectures: centralized and distributed. Both can fit your needs depending on your team size, business, and processes.
A centralized version control system retains a central copy of the project in a single repository and commits changes to that central version. Centralized Version Control Systems are based on a client/server architecture.
Every developer on the system has access to information about the work of other developers working on the project. It is easy for beginners to deal with a centralized version control system because it is very similar to 4D Server with binary databases.
While developers can work concurrently, the system only publishes the latest version of the file. Issues can take place if two or more developers simultaneously make changes.
The most common centralized version control system is Subversion, aka SVN.
A distributed version control system has different functional elements for collaborative editing and sharing, which allows each developer to manage their own repository. Distributed Version Control Systems use a peer-to-peer approach to version control, allowing developers to work productively when not connected to a network. This system works to avoid stepping on or overwriting others’ work and provides document history preservation and reversion capabilities.
The most common distributed version control system is Git.
Which one to choose?
The choice depends on your comfort with VCS concepts.
Centralized VCSs are probably the closest setup to what you already know with binary databases on 4D Server.
Distributed VCSs have been the most widely used for many years, and many developers moved from centralized to distributed. So even if you’re not familiar with VCSs, you can choose a distributed one. You’ll be able to use it as a centralized VCS (you can determine a central repository that will be your source code reference).
For VCS newbies, here are three basic actions that are used in all VCSs, and that we mention later in this post:
- A Commit is a set of modifications stored in the repository. A commit can concern one or several files. Each commit is stored with information about who did what and when, with additional comments. Commits are the base element of traceability.
- Pulling from a repository is the action of getting source code from another repository (generally the central one) to put it on your own repository.
- Pushing to a repository means uploading local repository content to another repository (generally the central one).
This chapter will show you the benefits of using a VCS, depending on your programming habits.
As mentioned in the introduction, the benefits of using a VCS is not limited to development teams. Just because you develop applications alone doesn’t mean you don’t want to follow industry standards!
The first advantage is traceability. You already know who made changes to your source code —it’s you—, but knowing exactly what, when and why is very useful, especially when you’re trying to retrieve the information years after you’ve made the changes!
The second advantage is security. Every source code step is saved, as you commit your changes regularly, and every commit can be undone. Each VCS allows you to retrieve changes made during a period of time or retrieve changes made to lines of source code. A big time-saver!
Sample of repository:
And for additional security, you can define a remote backup repository to avoid hardware problems on your computer. It’s up to you whether this remote repository is placed in another computer or server, in a private or public cloud, or a service.
Setting up development teams on binary databases with a 4D Server was an elementary task, but this simplicity had a cost. There were some limitations on the development process!
Everyone working in that environment encountered locks of methods, forms, or resources by another team developer. Usually, the work had to be clearly defined for each developer in order to avoid these locks.
But some resources are often used by all developers in the team. It’s the case for the structure editor, the toolbox (for menus, style sheets, and so on) or database methods. And the list is not exhaustive!
Who hasn’t tried to add a variable in a compiler method, just to realize that another developer had already opened that method?
With a VCS, you work on your own computer, so it doesn’t matter if your colleague has opened the same compiler method on his or her computer and has gone for lunch. When you push your code, the VCS will automatically identify your modification and ask you what to do if your colleague has touched the same code lines.
Teams of developers working on binary databases also often encounter issues when modifying low-level methods. When a developer working on a low-level method wants to test it, the method is saved. This version of the method is immediately used by all the other developers, even if it’s not finished yet, sometimes generating errors (even blocking errors) on their computers.
With a VCS, there’s no risk of affecting your colleagues’ work, because you can work and test on your computer, and then push the source code on the shared repository when you’re finished. It offers better shareability.
The main benefit of these two examples is that each modification doesn’t affect other people’s work until it is shared with them. Now it’s the developers who decide when the source code must be shared with the team, not 4D.
Development outside the office
Using a VCS allows you to work locally on your computer. It doesn’t matter if you’re at the office, at home, on the train, or on the beach!
When you’re outside the office, you no longer need to use a remote desktop session to develop with 4D Server, and you no longer need to be online.
You can now work offline and push your tested modifications on the shared repository when you’re back online. Moreover, as the VCS pushes only modifications, it’s faster than synchronizing the whole binary database, and less bandwidth is required.
Developing and debugging server-side code
Sometimes, you want to debug code on the server-side, such as triggers, stores procedures, or methods executed on the server. To enable this, 4D allows a remote 4D connected to a 4D Server running on the same computer session to modify the source code. In this configuration, the remote 4D handles the local source code files directly, instead of the downloaded 4DZ file.
Having each developer working with their own remote/server application is much safer than a whole team modifying the same shared source code on a binary database with 4D Server. Indeed in this last case, problems described above (Team-based development) are likely to occur.
To test server code with a binary database, developers often needed to first add a TRACE command in the method executed on server, then connect to the development server with a remote desktop session, and finally debug directly on that server.
And what about the developer who has left for a meeting or lunch and left the method opened on his or her computer, blocking all the other developers? This real-life situation cannot be encountered when using a VCS and developing locally. With projects, developers are autonomous and can develop and test without impacting the work of other developers.
Using a 4D Server for development
Sometimes, you need to use a 4D Server for development. Not for managing source code (it’s now handled by the VCS), but to serve data that can be too huge to be used locally. To verify ongoing developments and check that the modifications perform the expected result, a reference data set is often used on development servers.
To allow this, you need to go through a few steps:
- Push your modified code on the shared remote repository.
- You only need to pull the remote repository to the local repository of the test server (manually or by calling a method executed server-side, which will use LAUNCH EXTERNAL PROCESS, for example). This way, the files in the project folder of the server will contain your modifications. But it’s not sufficient because 4D Server keeps the image of the source code in the folder upon starting.
- Refresh the source code in the 4D Server of the test server by moving the application to the foreground or using the RELOAD PROJECT command.
- Connect your 4D remote and test!
Using a 4D Server for testING
Using a test environment before deploying an application is a best practice. Using a 4D Server with projects for testing purposes is very similar to doing it for development.
Once the test server repository is connected to the shared repository, you can pull the repository, call RELOAD PROJECT to make sure the 4D Server is up to date, and launch the test procedure. Simple, isn’t it?
In any case, it’s simpler than working with binary databases. Back then, you needed to stop the development server from copying the database on the test server. Another way to prevent the development server from stopping was to back it up and restore the test server’s database, and it was much more complicated to set up!
Changes on AN interpreted 4D Server in PRODUCTION
Sometimes, a 4D Server is used with binary databases executed in the interpreted mode in a production environment. Typically, developers encounter this use case when they want to quickly perform changes in production (add a feature, fix a bug). Such actions must be performed with the highest precautions, as each source code modification in production will directly impact… in production. Which can be undesirable if such changes have not been thoughtfully tested in the first place.
When a developer adds changes to the source code, all the remote 4D of the production environment are in an unsafe logic state because they execute processes and forms designed to call the previous version of the code currently modified. For example, when a developer modifies parameters on a method, every remote 4D calling this method may encounter errors until the method calls have been changed too!
The 4D development environment doesn’t allow modifications to the server source code unless it runs on the same machine as the server, which is much safer! As developers work on their own computers, there’s no risk of destroying the production environment. They can now finish and test the modifications before delivering them to the production 4D Server. Each set of changes is a coherent whole, which can be safely sent to the production server.
To set up such an architecture with projects, you have to follow these steps:
- Push your modified code on the production repository.
- You only need to pull the remote repository to the local repository of the production server (manually or by calling a method executed server-side, which will use LAUNCH EXTERNAL PROCESS, for example). This way, the files in the project folder of the server will contain your modifications. But it’s not sufficient because the 4D Server keeps the image of the source code in the folder upon starting.
- Disconnect the 4D Remotes impacted by the code modifications.
- To refresh the source code in the 4D Server of the production server, just move the application to the foreground or use the RELOAD PROJECT command. The source code on the server-side now contains modifications.
- Reconnect the 4D Remotes disconnected at step 3.
4D has developed Project mode to answer major needs raised by many 4D developers, centered around managing source code’s traceability, security, and shareability. These needs couldn’t be answered using the binary format. With projects, 4D allows you to use version control systems that are standard, modern, constantly evolving and frequently used in the software engineering industry.
But keep in mind that project mode in 4D is not only dedicated to VCSs. It improves how you approach development, paving the way for object-oriented development with classes, form macros, native Silicon compliance, and many other great features to come!
In this post, we described how to answer use cases when you want to transition from binary databases to projects and have little knowledge of version control systems.
We’ll see in another post how to set up repositories adapted to specific situations and development modes, and we’ll present another great feature of VCSs: branches.
More general information on VCSs can be found on the following websites:
4D Blog posts about VCSs:
- 4D Project: Embrace a new era of collaboration
- Project databases: Git. Commit. Pull. Push & more
- Server code development in project applications
- gitignore and 4D: a useful feature for Git users
- How to share your 4D code via Github
4D and Git breakout session by Ahlam Benhachem during 4D Summit 2020: