howto://use TMG 2010 as the Exchange edge transport server

In our earlier posts (part one, part two) we performed a simple Exchange server setup for a ‘single-server’ install. We have our addressing policy setup, and can send email out to the world. Now, it’s time to get email IN to our system. Since we use TMG at our perimeter, we’re going to leverage the combined power of TMG and the Exchange 2010 Edge Transport role. Here’s a quick pic of what we’re going to have when we’re done.

We have our existing TMG2010 server running Windows 2008R2, and our Exchange 2010 server running Windows2008R2. We’ve already installed Exchange 2010 on Demeter, with the Client Access Server, and Mailbox Server roles. We have a mailbox database, mailboxes, and can send outgoing mail just fine. Now, we want incoming mail, and that will have to go through our TMG server.

We do have some pre-work to do, and if you opted for 2008 instead of R2, there will be a little more work to do, but overall, this is pretty straight-forward process, and much quicker than our ‘single-server’ install. Ready to begin?

DNS records

If we want external domains to send us mail, and to minimise the problems we’ll have sending mail out to external domains, we need to get our DNS work out of the way first. Depending upon the TTLs set, we may finish the rest of this before DNS is done, so we want to do this first. We need to log onto our DNS server (or hosted portal) and setup our records for our domain. This means we need to

Created an A record for our host, Demeter, using our external ip.addr.
Created an MX record with a weight of 10, and pointed it to our Demeter.
Updated our SPF record to include Demeter as a valid sender by ip.addr. See this post for more on that.
Update our PTR record if we have access to it.
After you’ve allowed time for DNS changes to propagate, use a looking glass or dig to make sure your A, MX, SPF, and PTR (optional) records are all set.

Yes, I left the googlemail servers in place, since I originally setup retrohack.com on Google Apps. If I lose Internet connectivity at the house, I want to still receive mail, so those servers will accept mail that I can fetch later.

PowerShell

We’re running our TMG 2010 server on Windows 2008 R2, which means we already have PowerShell 2.0. That’s good, because Exchange 2010 requires it. If you are using the original version of 2008, and added PowerShell through Server Manager, you’ll need to remove it. Here’s how.

Click Start, Run, and type CompMgmtLauncher. Click OK.
In the Server Manager tree, expand Features, and in the Features Summary, click Remove Features.
In the Remove Features Wizard, scroll to Windows Powershell, and then clear the check box.Click Next, and then click Remove.
When the process has completed, restart the computer

When you start the install of Exchange 2010, the prerequisites page will direct you to install PowerShell 2.0.

AD LDS, or the service formerly known as ADAM

Since we’re dealing with a role that is intended to live in the DMZ, and not have any connectivity to Active Directory, we’re going to need to run Active Directory Lightweight Directory Services on our TMG server, so that the Exchange 2010 Edge Transport Role has a place to store its configuration. I know, that seems awfully silly to me to, seeing as how a flat config file, registry hive, or even an XML file would do just fine. But they paid a lot of money to develop ADAM/AD LDS, so by gods somebody is going to use it! To install Active Directory Lightweight Directory Services. If you have already installed TMG 2010 you already have AD LDS running, and are good to go. If you’re not to the install of TMG yet, here’s a quick way to get AD LDS on board.

From an elevated command prompt, type cmd.exe /c start /w pkgmgr.exe /iu:"DirectoryServices-ADAM".

Installing TMG 2010

As we’re dealing with adding Exchange 2010 to a setup that already has TMG 2010 in place, we won’t cover that install in this post. If you need to, you can see this two part post on installing TMG 2010 by clicking part one, and part two. Come back here when you’re done, and we’ll continue. It’s okay, we’ll wait. You go right ahead.

Installing the Exchange Server Edge Transport role

Before installing the Exchange Server Edge Transport role, you must verify that the computer is configured with a DNS suffix. Use the following procedure:

To add a DNS suffix to a Forefront TMG computer
On the Desktop, right-click Computer, and select Properties.
Click Advanced system settings, and then click the Computer Name tab.
Click Change, and then click More.
In the Primary DNS suffix of this computer box, if an FQDN is not configured, type one, and click OK.

To install the Exchange Server Edge Transport role,

Run the Exchange Server Setup.exe file, and follow the steps in the Exchange Server Setup Wizard, including the installation of all the prerequisites.
On the Installation Type page, click Custom Exchange Server Installation.
On the Server Role Selection page, select Edge Transport Role, and click Next.
On the Readiness Checks page, view the status to determine if the organization and server role prerequisite checks completed successfully.
Then, click Install to install Exchange.
Go get coffee…mmm, coffee…
On the Completion page, click Finish.
When the EMC launches, select the Edge Transport Role, and select "Enter Product Key…" from the Actions pane.
On the Edge Transport server, open an Administrative Exchange Management shell, and run the following command. New-EdgeSubscription -FileName "C:\EdgeSubscriptionInfo.xml"
Copy the resulting XML file to a Hub Transport server in the Active Directory site to which you want to subscribe the Edge Transport server.
On the Hub Transport server, open an Administrative Exchange Management shell, and run the following command.
New-EdgeSubscription -FileData ([byte[]]$(Get-Content -Path "C:\EdgeSubscriptionInfo.xml" -Encoding Byte -ReadCount 0)) -Site "Default-First-Site-Name" -CreateInternetSendConnector $true -CreateInboundSendConnector $true
note to readers: if your site is NOT named Default-First-Site-Name, change that as necessary in the above command!

note to Microsoft: if you were trying to make this as difficult as Unix cmd-line administration, you succeeded. Seriously, you really expect someone to remember this sequence? You’re own page on TechNet where I got this from has typos!!!
On the Hub Transport server, run the following command.

Start-EdgeSynchronization
Launch the TMG Management Console.
Come down to E-Mail Policy and click Configure E-Mail Policy to launch the wizard.
Click Next on the welcome screen, then add the internal mail server and accepted authoritative domain(s) as necessary, and then click Next.
Configure your internal listener. Unless you are multi-homed on the internal network, you can probably just click Internal and then click Next.
Configure your external listener ip.addr, and the FQDN your TMG server will use in response to HELO/EHLO messages.
Enable as desired. You definitely want to enable connectivity for EdgeSync since that is how we’re rolling.
And of course, it’s going to make sure you understand that you’re opening stuff up. Click Yes, unless of course, you don’t really want email to work.
Hit apply, enter your configuration chance description, and go for the win!
Then, go to http://www.mailradar.com/openrelay/ and test your server to make darn sure that you are not relaying at all! Do not skip this step, as it’s the best way to make sure you didn’t make a misconfiguration that leaves your server as an open relay. Failure to do this can result in your ISP to blocking you, Santa placing you on his naughty list for life, and shrinkage. Don’t be that guy.

If all was done per the specs above, you should be ready to rock and roll! If not, start troubleshooting from the outside in. Use your Gmail account to send email, and run WireShark on your TMG server to make sure traffic is getting past your ISP. If it’s not, start by making sure your DNS records are correct, and have propagated, then make sure you can get traffic past your ISP (not all will allow SMTP in by default, it at all.) If email is getting to your TMG/Edge server but not past it, use the Queue Viewer Tool (Exchange Management Console, Tools, Queue Viewer) and check for error codes. Exchange authentication needs to use internal FQDNs or NetBIOS names. See this post for the most common error there.

Now that you’ve got email working, take a break and loosen up gray matter with this little nugget from the tubes…YouTube that is. StrongBad rules, and with TMG, we fear no virus!