Update (2015-06-30 ~12.00 UTC):
+The replacement buildbot master is now live. The CMS service and the ci.apache.org website have been restored. The project CI builds are mostly working but builds that upload docs, snapshots etc. to the buildmaster for publishing are likely to fail at the upload stage while we ensure all the necessary directory structures are in place to receive the uploads. Work to resolve these final few issues is ongoing.
We continue to try and contact the owner of the account where the IRC proxy was running. In case their account has been compromised, it remains locked. In addition, all their commits have been reviewed by other project committers and that review has comfirmed that no malicious commits have been made by the account in question.
+The review of aegis.apache.org is ongoing. No evidence of compromise beyond the possible compromise of the single, non-privileged user account has been found.
Original post (2015-06-29 ~21.00 UTC):
+As per the e-mails to committers@ earlier today, aegis.apache.org is currently offline after a report was received that suspicious network traffic had been observed from that host. This blog post will be updated as more information becomes known.
+What we know:
+It remains unclear whether the open IRC proxy was installed by the user that owned the account or whether their account was compromised and the IRC proxy was installed by an unauthorized user.
It is worth stressing that no further information came to light between 20.00 UTC 28 June 2015 and 10.00 UTC 29 June 2015 that triggered the decision to take the host off-line. The host was taken off-line purely as a precaution while we reviewed the available information. That process is ongoing. So far we have found no evidence to even suggest anything more than a user account being used to run an IRC proxy and plenty of evidence that suggests that this was the only activity this account was used for.
Risks:
+There is no risk to released source or binaries for any ASF project. There are multiple reasons for this:
+Buildbot is used to build some project web sites and / or project documentation. The risk of compromise here is viewed as very low for the following reasons:
+Project impact:
+The following services are currently off-line and will remain so until the buildbot master is restored
+Work in progress:
+Analyzing aegis.apache.org is going to take time and, while we view the chances of a wider compromise of this host as very, very small, we are not willing to bring the host back on line at this point. This host was due for replacement so the decision has been taken to pull this work forward and rebuild the buildbot master on a new host now. We have taken this decision not because we believe aegis.apache.org to be compromised, but because it is possible to complete this work far more quickly than it is possible to confirm our view that aegis.apche.org is not compromised. We currently estimate that the rebuild of the new buildbot master host will be completed by 1 July 2015.
We continue to analyze the information we have obtained from aegis.apache.org and from other sources and will update this blog post as more information becomes available.
+Questions:
+Questions, concerns, comments etc. should be directed to infrastructure@apache.org