The Core Technologies Blog
Our Software // Windows Services // 24×7 Operation
Recently, a few of our customers running Dropbox as a windows service with AlwaysUp have reported failures after manually updating to the latest version of Dropbox. These are happening because Dropbox changes the location of its executable file during the update. Let’s dig into the details and see how to fix the problem.
The Old Dropbox Location
Prior to version 3.5.58, released in April 2015, Dropbox stored its application executable file (Dropbox.exe) in a user-specific folder. If you installed Dropbox in the Administrator account, you could find the file in C:\Users\Administrator\AppData\Roaming\Dropbox\bin:
The New Dropbox Location
If you install a fresh copy of version 3.5.58 or later, you will notice that Dropbox.exe now lives underneath the Program Files folder, in C:\Program Files (x86)\Dropbox\Client (or C:\Program Files\Dropbox\Client on 32-bit versions of Windows):
We have already noted the new location in our step-by-step tutorial to guide future customers, but the story is more complicated for existing installations…
Upgrading Dropbox Changes its Location (and Breaks your AlwaysUp Configuration)
Unfortunately the impact of this change to the file locations isn’t limited to new installations. Manually updating an existing Dropbox installation will cause the files to be moved to the new folder. This isn’t a problem for Dropbox itself, but since AlwaysUp is blissfully unaware of the change, it will continue to run Dropbox.exe from its old location (as you originally specified in the Application field on the General tab):
Because the executable is no longer present there, AlwaysUp will fail to start Dropbox and report errors like this:
The Fix: Set the New Dropbox Location in AlwaysUp
Fortunately normal operation can be restored quite easily. Simply edit Dropbox in AlwaysUp and update the Application field on the General tab to specify the new path to the Dropbox executable (C:\Program Files (x86)\Dropbox\Client\Dropbox.exe):
Save your settings and AlwaysUp should be able to start Dropbox normally again!
PC Commander 4.0, our free tool that helps you manage your remote computers, is now available for download. This convenient software allows you to control your PC entirely from your web browser. Google Chrome, Mozilla Firefox, Internet Explorer and even Microsoft’s new Edge browser are all supported.
Here are a few things you can do with PC Commander:
Securely access your PC
Type in the URL and specify your software-configured password to login to your computer. Note that you can setup PC Commander to use HTTPS to ensure that all communications are encrypted.
Understand what is running, restart your PC and more
After you login, the System page will present a general overview of what is running on (and how many user are logged on to) your computer. From there you can even restart your server — a very convenient feature if the PC is hung or cannot respond to an interactive login!
Manage all processes
The Processes page lists everything running on your computer. Easily see who started each process and how much CPU and memory is being consumed by each one. Click on a column’s title to sort the table by that column — very useful when looking for memory or CPU “hogs”!
Stop or modify any process
From the table of processes, click a name to “drill down” into a single process. From there you can stop the process or change its priority to give it more or less resources on the computer. You can also quickly find some information about the process at ProcessLibrary.com if you don’t recognize it…
Manage all Windows Services
Go to the Services page to see all windows services installed on your PC. The table shows which services are running (via the green circle on the left), how services are started and the user account that runs each service. Columns are sortable for your convenience and you can easily start, stop or restart any service directly from the table.
Start, stop or restart any Windows Service
Click into a specific service to see additional details about it. The full command line used to launch the service is shown, and all dependent services are identified. You can easily start, stop or restart your service from this page.
View all users logged in
The Users page lists all users currently logged on to the computer. With a couple of clicks, you can quickly disconnect an account to allow another user to log on.
Log off, disconnect or send a message to a user
Drill down into a specific user entry to see additional options. Send a message directly to a user’s screen to notify him that you are about to log him off!
Browse your hard drive and view/download any file
Open the Drives page to list all storage devices attached to your computer. Navigate to any folder and view or download any file you like! And you can see detailed information on a file by clicking on its name.
View the Windows Event Logs
The Event logs page comes in handy when you want to investigate system errors or see what your windows services are up to. Click on a log to review its detailed records in a friendly tabular format.
And that’s it! We hope that you will find this free software useful when managing your computers. Please be sure to get in touch if you have ideas on how we can make PC Commander even better for you.
We all know to fire up the Windows Task Manager when we want to terminate a process, see how much memory is being used or find that rogue program hogging the CPU, but do you know that Task Manager also shows important details of Windows Services? For some situations, the Task Manager, with its focus on what is actually running on the PC, is even more useful than the standard Services Control Panel!
Information about your PC’s services can be found in the Task Manager’s Services tab. If you don’t see any tabs across the top of the Task Manager window, click the More details link at the bottom to reveal them:
The services table shows all the windows services installed on your PC. Helpful columns show the description, status and group of each service. And, perhaps most importantly, services that are running will show the numeric process identifier (PID) of the process hosting the service. That PID is useful when you want to terminate an unresponsive service, boost its priority or perform other low-level operations.
Right-click an entry to summon the context menu offering several operations you can perform on the service:
Start, Stop or Restart your Windows Service
Take action on your service right here! No need to open services.msc, or run the NET START or NET STOP commands from a prompt.
Selecting this entry will open the Windows Services Control Panel. Note that you can also open Services by clicking on the Open Services link at the bottom of the window.
Choosing this entry opens your browser and searches for the service’s name and display name. For example, if you have selected the service named Spooler with display name Print Spooler, selecting “Search online” will open your browser and launch a convenient Bing search (or Google, if that is your default search engine) for Spooler Print Spooler:
Go to details
Selecting this entry, which is only available when your service is running, switches you to the Task Manager’s Details tab with the service’s underlying process highlighted.
For example, here is the VirtualBox service (VBoxService) running with PID 624 as we select “Go to details”:
We are immediately transported to the Details tab to provide us with more information on the process with PID 624 (named VBoxService.exe):
As you can see, Task Manager provides some decent tools for controlling your windows services. Be sure to fire it up next time you have to work with a service!
Wellsoft Corporation, developer and provider of the industry-leading Emergency Department Information System (EDIS) since 1990, is consistently ranked #1 in KLAS user surveys of EDIS time and again. Recognized for providing the most complete functionality and most configurable product on the market, Wellsoft is the specialist in Emergency Department Information Systems.
Wellsoft has been including AlwaysUp for its clients since 2007.
Why does Wellsoft use AlwaysUp for all implementations?
Wellsoft’s Emergency Department Information System (EDIS) is a mission critical application for hospital emergency departments. The software must operate 24 hours a day, with unparalleled availability and reliability. AlwaysUp helps us maintain superior uptimes for clients by:
- Automatically starting the Wellsoft application if the server is rebooted
- Automatically restarting the application if the server hangs or uses too much memory
- Restarting instances of Wellsoft during routine Microsoft memory or CPU spikes that close a running copy
- Sending email alerts to report potential problems and support our IT staff
We selected AlwaysUp based on its feature set, simplicity and cost — it was the best solution for our clients.
How are you using AlwaysUp today?
AlwaysUp is a required part of every implementation of Wellsoft EDIS. To date, we manage over 70 licenses of AlwaysUp in various versions.
What are your favorite features of AlwaysUp?
For Wellsoft: The ease of use, immediate notifications and customer service are right at the top of the list. When we do require assistance, the level of support from Core Technologies is unrivaled.
For our clients: Ease of use; we have found that even our least “techy” clients can use the information provided by AlwaysUp to determine what they can handle and when they need to involve Wellsoft.
Any other comments on working with our company, support, etc?
Yes! We recently used the Activity Report feature to help us locate a memory issue that was causing several of our clients to experience program disruptions. We were able to narrow our focus thanks to AlwaysUp’s weekly statistics, saving us incalculable time.
Our month-long evaluation of AlwaysUp 9.1 running windows services on Windows 10 started soon after the new operating system was released on July 29th. Testing was comprehensive, targeting all major features including:
- Installation & setup
- Sending emails in response to key events
- Running programs in the background in Session 0
- Starting applications interactively in the current session
- Restarting an application if it uses too much memory
- Detecting CPU spikes
- Responding to customized failure detection scripts (“sanity checks”)
- Automation in the current session with AutoIt
- Interaction with other windows services installed on the PC
… and much more.
We also focused on running popular applications that have already earned compatibility with Windows 10.
Dropbox continued to synchronize files as normal when run as a windows service:
And Google Drive operated flawlessly as well:
The end result is that AlwaysUp 9.1 is fully compatible with Windows 10!
The only trouble we noticed was when interacting with applications running in Session 0…
The Keyboard and Mouse didn’t work in Session 0
We were able to switch to the isolated Session 0 but couldn’t interact with the application running there. The mouse cursor vanished and the keyboard was unresponsive. We couldn’t even click the “Return now” button to go back to the normal desktop!
Fortunately this bug has already been reported to Microsoft and a resolution may be on the way.
We were unable to see Session 0 when logged in through RDP
The second oddity we encountered was when logged in to our server via RDP. Switching to Session 0, either from the Interactive Services Detection Dialog or from AlwaysUp, resulted in an unhelpful “black screen”. The black screen remained for one minute, until the Session 0 timeout elapsed and we were unceremoniously disconnected from the server. We experienced this with RDP versions 6.2.9200 and 6.3.9600 and our recommended driver updates didn’t resolve the problem. Hopefully Microsoft will fix this one quickly too.
September 19 2015: Switching to Session 0 over RDP is now working on our Windows 10 machines! We think that the cumulative update for Windows 10 issued on September 8 (KB3081455) may have done the trick so please apply that update if you are still having trouble.
Trinium Technologies provides business software for two primary verticals: intermodal trucking companies and fuel marketers. Its applications enable companies to reduce cost, grow revenue, and improve customer service. Customers range in size from small businesses to large enterprises with both cloud services and on-premises deployment options available.
Barry Assadi, the Chief Technical Officer at Trinium, has been using AlwaysUp since 2006. He knows AlwaysUp very well and was interviewed for this article.
Why does Trinium use AlwaysUp?
Trinium uses AlwaysUp to handle processes for file transfer (FTP), file import & export interfaces, and report generation. We used to deploy automated background processes with Windows Task Scheduler, but there were constant stability issues along with a lack of visibility of the status of these processes. AlwaysUp runs our executables without fail, and its detailed reporting tell us how our applications are doing.
Customer service is very important to us, so we have a dedicated AlwaysUp process per customer that is used to keep a heartbeat on all of the services we’ve created. This way we can see if any processes are hung or stopped to avoid any downtime for our customers.
How is AlwaysUp deployed in your business?
Each non-hosted customer will get an installation of AlwaysUp to run our software as a service in their environment. We also run AlwaysUp in Trinium’s hosted private cloud, which has a few dedicated servers reserved for processing using AlwaysUp.
What are your favorite features of AlwaysUp?
Over the years, we’ve come to appreciate the depth of settings to manage the processes we model as a service. There seems to be a solution for everything we’ve thrown at AlwaysUp!
Our more technical customers are happy that they can now stop and restart our applications from their batch files (using the NET command).
And we especially like the ability to multi-select to start/stop/restart processes. It is very convenient as our techs install and work with multiple services per server.
Incorporated in 2007 and driven by new opportunities in the “Internet-of-Things” (IoT) market, CartaSense offers end-to-end monitoring and alerting capabilities for cold-chain, agricultural and asset security applications. CartaSense provides complete field proven systems using wireless sensors, gateways and cloud based computing that deliver real-time information to support real-time decision making for increased efficiency and profits.
Chaim Belfer, the Vice President of R&D at CartaSense, has been using AlwaysUp with his Java programs since 2010. He currently manages more than 20 installations and was interviewed for this article.
How does CartaSense use AlwaysUp?
CartaSense deploys several Java applications that manage a world wide network of sensors. For security and management purposes, we need to run our windows installations as windows services. Looking for a solution we found AlwaysUp which solves the problem — plus helps with recovery and logging.
For the past few years, AlwaysUp has been included whenever we install our software on a Windows Server (2008 or 2012). We also recommend AlwaysUp to our own customers.
Why did you choose AlwaysUp for your Java applications?
We found your company through a Google search a few years ago. I tested several open source solutions but none of them really did the job, or at least needed a lot of development to make it work reliably. AlwaysUp worked well from the beginning (the 30-day trial) so we started out with a single license and bought more over time, as needed.
What are your favorite features of AlwaysUp?
Beyond solving the big problem of running our Java applications as windows services, AlwaysUp’s GUI is very nice and self explanatory. The documentation is also good, and the software is easy to use. Prompt service and quick response to our support emails is a big plus too!
First, let’s start with some definitions…
What is a regular application?
A regular application is one that you start and run on your Windows desktop. You launch it from a desktop icon, by double-clicking an associated document, or by typing its name at a command prompt. Examples include Microsoft Excel, Adobe Reader, Google Chrome and virtually any other program you consciously interact with on your PC. These are the programs that you know, love and use every day — the ones that make your computer a remarkably powerful tool.
What is a windows service?
A Windows Service is a special program designed to “plug into” Windows. It typically chugs along doing its work in the background, never revealing itself, with little or no attention from anyone using the PC. It turns out that a whole host of these invisible workers are necessary to make your PC work smoothly! For example, the Print Spooler windows service ensures that your documents arrive at the printer, while the Workstation service makes the Internet available for all applications. Important tasks indeed!
What’s the difference?
So with those basics in place, here are the 10 most important ways that a windows service differs from a regular application:
A windows service can start automatically at boot, without anyone having to log on. A regular application only runs when a user is logged in. This key distinction makes the windows service framework the better choice for software that must run 24×7, surviving the inevitable reboot along the way.
A windows service won’t exit when you or anyone else logs off the PC. A regular application will stop when you log out.
Windows services don’t run visibly on your desktop (not since Vista). They are confined to the isolated Session 0 and can not (usually) show their windows or tray icons to anyone logging into the PC. This restriction makes it very difficult for a windows service to interact with someone using the computer and consequently Microsoft strongly discourages implementing GUI services.
A windows service can be easily started or stopped from the command line using the NET or SC commands. This capability is a boon to sysadmins and technical folks, who can conveniently manage a service’s lifetime through unattended, administrative scripts.
Only a single instance of a windows service can be started at any time. That single copy must serve all users on the PC. This is very different than a regular application, which typically permits multiple copies, especially when several people are logged in.
Services typically run in the Local System account, which has administrative powers over many aspects of the PC. Thus your average service will have more rights and control over your computer than your typical application running in the context of a regular user’s account.
When run as LocalSystem, a service has access to UNC paths (for example, “\\server\data”) but cannot access mapped drive letters. You should run the service in a regular user account to access network drives, and even then the service may have to map drives explicitly (for example, with the NET USE command) to gain access.
All windows services support sophisticated recovery options for when they stop unexpectedly. You can restart the service, run a command line or even reboot the computer.
While each regular application is backed by a single, unique process, it is possible for many services to share a single running process. This situation makes it possible for some services to start rapidly and make efficient use of resources — important characteristics for critical components that must always be available.
A windows service must contain special code to respond to instructions from the Windows Service Control Manager (SCM). For the technical folks, this means implementing the ServiceMain (or equivalent) entry point in your application.
AlwaysUp is designed to run your application as a Windows Service and automatically restart it when it crashes or stops for any reason. Even though this frees you from having to take any action on a failure (eliminating those unwelcome “server-not-working” calls at 3 AM), you may still want to:
- Find out how often your application is failing
- Know the time when your program stops working (for diagnostic purposes)
- See how your application is using CPU and memory over the course of the day
- Find out whenever your server has been rebooted
Automatic email from AlwaysUp will tell you all that, and more!
Specifying Who to Email & When to Send a Message
To setup email in AlwaysUp, edit your application and switch to the Email tab:
Enter the following:
- The email address to notify. To send to multiple addresses, just separate each one with a comma (,).
- When AlwaysUp should send you an email — either in response to an important event, or at a regular time. The daily and weekly summary emails are a good way to monitor your application’s activities without having to receive multiple emails throughout the day.
- A line of text to be included in the email. This value is optional, but you can use it to explicitly label a server & application when you have many installed with the same name (for example, when you are running the same application on many different workstations).
AlwaysUp doesn’t require that you have Outlook or any other email software installed. Instead, it deals directly with your SMTP server – the machine actually responsible for routing your email over the Internet and delivering it to your recipients. Every major provider (such as Gmail, Yahoo and Office 365) supports SMTP mail and it is simply a matter of providing those settings to AlwaysUp so that it can send messages on your behalf.
Click the Configure… button to reveal the details:
The Configure Email Settings window that comes up requests quite a bit of information, but don’t be scared! Your systems administrator will be able to fill this in while balancing a bowl of water on his head and dancing the Macarena, but in case you are entering these values yourself here is what each field means:
- Address: The email address that will be displayed in the “From” field of the email messages sent by AlwaysUp.
- Name: The name that will be displayed in the “From” field of the email messages sent. This value is optional.
- Name/IP: The name (or IP address) of your mail server.
- Port: The numeric port on which the mail server is available. Note that the default is 25 for regular SMTP servers and 465 for servers using SSL – please specify one if these if in doubt.
- This server requires encryption: Check and make the appropriate selection if the mail server uses TLS, SSL or STARTTLS.
- This server requires authentication: Check if the server requires authentication, and select the authentication method. The choices are CRAM-MD5, AUTH-LOGIN, AUTH-PLAIN, and AUTH-NTLM. A login and a password will be required for all but AUTH-NTLM.
- Login: The user name/login for the mail server. Usually your full email address.
- Password: The password for your email account.
How to use your Gmail Account
To send email with your Gmail account, please specify the following settings:
||Your full gmail address (email@example.com)
|SMTP Server Name/IP:
||Your full gmail address (firstname.lastname@example.org)
||Your gmail password
So for someone named Mike Jones, whose Gmail address is email@example.com, the Configure Email window should look something like this:
How to use your Microsoft Office 365 Account
If you have an Office 365 mail account, please specify the following settings:
||Your full Office 365 email address
|SMTP Server Name/IP:
||Your full Office 365 email address
||Your Office 365 password
So for Mike Jones, whose Office 365 email address is firstname.lastname@example.org, the Configure Email window should look something like this:
Using Another Email Service
This list documents SMTP settings for many other email providers.
So that’s it! Please be sure to get in touch if you have any trouble setting up automatic email from AlwaysUp.
Crashes and memory leaks are particularly offensive when the occur in windows services – applications designed to run unattended, 24/7 in the background to perform important tasks. Our AlwaysUp and Service Protector utilities go a long way to diminishing the effect of these insidious failures, but using them is no substitute for finding and fixing a problem at its root.
Developers looking to diagnose failures in their windows services should consider using Microsoft’s Debug Diagnostic Tool. This free desktop application, nicknamed DebugDiag, will monitor your windows service process and create a “dump” describing the state of the application when it crashed (or started using too much memory). You can use this information to narrow down the problem in your own code, or pass it on to the folks who developed the failing service for their expert analysis.
Using the Debug Diagnostic Tool on your Windows Service
Download the Debug Diagnostic Tool from microsoft.com.
Install it. Setup should be quite straightforward, with the usual prompts for a folder, etc.
From Windows Explorer, navigate to the folder where DebugDiag was installed (C:\Program Files\DebugDiag by default). Start DebugDiag.Collection.exe – the application that monitors and collects information from a running process or service.
The Debug Diagnostic Tool window should come up and you will be launched immediately into setting up a new “rule” describing what to watch for. From here, you can specify to monitor for crashes, leaks, and even performance-related issues. We’ll select Crash for this tutorial and click Next to move on.
Next, select the A Specific NT service option:
Select the service you wish to debug from the list. We chose “AA MyService”, a windows service we developed to test our Service Protector application. Click Next.
Almost there! Leave the Advanced Configuration settings at their default values and click Next to continue:
You don’t have to make any changes on the Select Dump Location screen either – just make a note of where the dump file will be saved.
And finally, let’s activate the rule now and click Finish:
After a few seconds, the tool will be monitoring your windows service for crashes:
Analyzing a Windows Service Crash
When the service crashes, DebugDiag will record a dump file and increment the Userdump count column. Here we can see that 1 crash has occurred and been captured (with the full path to the dump file noted on the right):
To dig into the dump file:
Start DebugDiag.Analysis.exe from Windows Explorer:
The DebugDiag Analysis window will open momentarily:
To configure analysis:
- Check the CrashHangAnalysis box near to the top.
- Click the Add Data Files button, navigate to the location of the dump file (it is mentioned in the Diagnostic tool we set up previously) and select it. A new entry should show up in the lower pane.
And click the Start Analysis button to get going! After a period of “thinking”, the application will open its report in your browser:
Scan the report for any smoking guns. Thread call stacks may be particularly useful. For example, this stack trace tells us that our service crashed while handling a button press. (This is not surprising though – that is what we did to force the crash in our testing!)
Hopefully you will find a telltale sign to help you identify and fix your problem.
Also Find Memory Leaks in your Windows Service
If your windows service is consuming too much memory instead of crashing, configure the collector to watch for memory leaks:
The final report will detail all outstanding memory allocations.