Monday, December 3, 2007

Google's BigTable

官方的 Google Reader blog 中有对BigTable 的解释。这是Google 内部开发的一个用来处理大数据量的系统。这种系统适合处理半结构化的数据比如 RSS 数据源。 以下发言是 Andrew Hitchcock 在 2005 年10月18号基于: Google 的工程师 Jeff Dean 在华盛顿大学的一次谈话 (Creative Commons License).

首先,BigTable 从 2004 年初就开始研发了,到现在为止已经用了将近8个月。(2005年2月)目前大概有100个左右的服务使用BigTable,比如: Print,Search History,Maps和 Orkut。根据Google的一贯做法,内部开发的BigTable是为跑在廉价的PC机上设计的。BigTable 让Google在提供新服务时的运行成本降低,最大限度地利用了计算能力。BigTable 是建立在 GFS ,Scheduler ,Lock Service 和 MapReduce 之上的。

每个Table都是一个多维的稀疏图 sparse map。Table 由行和列组成,并且每个存储单元 cell 都有一个时间戳。在不同的时间对同一个存储单元cell有多份拷贝,这样就可以记录数据的变动情况。在他的例子中,行是URLs ,列可以定义一个名字,比如:contents。Contents 字段就可以存储文件的数据。或者列名是:”language”,可以存储一个“EN”的语言代码字符串。

为了管理巨大的Table,把Table根据行分割,这些分割后的数据统称为:Tablets。每个Tablets大概有 100-200 MB,每个机器存储100个左右的 Tablets。底层的架构是:GFS。由于GFS是一种分布式的文件系统,采用Tablets的机制后,可以获得很好的负载均衡。比如:可以把经常响应 的表移动到其他空闲机器上,然后快速重建。

Tablets在系统中的存储方式是不可修改的 immutable 的SSTables,一台机器一个日志文件。当系统的内存满后,系统会压缩一些Tablets。由于Jeff在论述这点的时候说的很快,所以我没有时间把听到的都记录下来,因此下面是一个大概的说明:

压缩分为:主要和次要的两部分。次要的压缩仅仅包括几个Tablets,而主要的压缩时关于整个系统的压缩。主压缩有回收硬盘空间的功能。Tablets的位置实际上是存储在几个特殊的BigTable的存储单元cell中。看起来这是一个三层的系统。

客户端有一个指向METAO的Tablets的指针。如果METAO的Tablets被频繁使用,那个这台机器就会放弃其他的tablets专门支 持METAO这个Tablets。METAO tablets 保持着所有的META1的tablets的记录。这些tablets中包含着查找tablets的实际位置。(老实说翻译到这里,我也不太明白。)在这个 系统中不存在大的瓶颈,因为被频繁调用的数据已经被提前获得并进行了缓存。

现在我们返回到对 列的说明:列是类似下面的形式: family:optional_qualifier。在他的例子中,行:www.search-analysis.com 也许有列:”contents:其中包含html页面的代码。 “ anchor:cnn.com/news” 中包含着 相对应的url,”anchor:www.search-analysis.com/” 包含着链接的文字部分。列中包含着类型信息。

注意这里说的是列信息,而不是列类型。列的信息是如下信息,一般是:属性/规则。 比如:保存n份数据的拷贝 或者 保存数据n天长等等。当 tablets 重新建立的时候,就运用上面的规则,剔出不符合条件的记录。由于设计上的原因,列本身的创建是很容易的,但是跟列相关的功能确实非常复杂的,比如上文提到 的 类型和规则信息等。为了优化读取速度,列的功能被分割然后以组的方式存储在所建索引的机器上。这些被分割后的组作用于 列 ,然后被分割成不同的 SSTables。这种方式可以提高系统的性能,因为小的,频繁读取的列可以被单独存储,和那些大的不经常访问的列隔离开来。

在一台机器上的所有的 tablets 共享一个log,在一个包含1亿的tablets的集群中,这将会导致非常多的文件被打开和写操作。新的log块经常被创建,一般是64M大小,这个 GFS的块大小相等。当一个机器down掉后,控制机器就会重新发布他的log块到其他机器上继续进行处理。这台机器重建tablets然后询问控制机器 处理结构的存储位置,然后直接对重建后的数据进行处理。

这个系统中有很多冗余数据,因此在系统中大量使用了压缩技术。

Dean 对压缩的部分说的很快,我没有完全记下来,所以我还是说个大概吧:压缩前先寻找相似的 行,列,和时间 数据。

他们使用不同版本的: BMDiff 和 Zippy 技术。

BMDiff 提供给他们非常快的写速度: 100MB/s – 1000MB/s 。Zippy 是和 LZW 类似的。Zippy 并不像 LZW 或者 gzip 那样压缩比高,但是他处理速度非常快。

Dean 还给了一个关于压缩 web 蜘蛛数据的例子。这个例子的蜘蛛 包含 2.1B 的页面,行按照以下的方式命名:“com.cnn.www/index.html:http”.在未压缩前的web page 页面大小是:45.1 TB ,压缩后的大小是:4.2 TB , 只是原来的 9.2%。Links 数据压缩到原来的 13.9% , 链接文本数据压缩到原来的 12.7%。

Google 还有很多没有添加但是已经考虑的功能。

1. 数据操作表达式,这样可以把脚本发送到客户端来提供修改数据的功能。
2. 多行数据的事物支持。
3. 提高大数据存储单元的效率。
4. BigTable 作为服务运行。
好像:每个服务比如: maps 和 search history 历史搜索记录都有他们自己的集群运行 BigTable。
他们还考虑运行一个全局的 BigTable 系统,但这需要比较公平的分割资源和计算时间.

Tuesday, October 30, 2007

Solaris 10 and Net-SNMP

Net-SNMP is provided in Solaris 10 release now, and how to use it is descripted in this article.

To disable Sun's SNMP
#svcadm disable svc:/application/management/snmpdx:default

the Net-SNMP daemon program locates at /usr/sfw/sbin/snmpd, and configure file of snmpd saved with the name snmpd.conf at /etc/sma/snmp. Use svcadm to start of stop snmpd:
#svcadm enable svc:/application/management/sma:default
#svcadm disable svc:/application/management/sma:default

Log File
if there's any error or warning messages, see the file /var/svc/log/application-management-sma\:default.log. The default log file of snmpd is /var/log/snmpd.log.

Configuration file
We may use /etc/sma/snmp/snmpd.conf.save as the working configure file to test wheather our snmpd daemon is ok.
#cp /etc/sma/snmp/snmpd.conf.save /etc/sma/snmp/snmpd.conf
#svcadm restart sma

P.S. There's another file also named as snmpd.conf stored in /var/sma_snmp directory, and which is called persistent storage file. The persistent storage file, /var/sma_snmp/snmpd.conf, contains USM security information and any MIB components that are set for persistent storage. This file also contains the engineID and the engineID boots. This persistent storage file is automatically updated when the System Management Agent starts. When the System Management Agent stops, the snmpusm and snmpvacm utilities write user security information to this storage file.

Using the AgentX Protocol
The AgentX protocol is supported in the System Management Agent. By default, the System Management Agent ships with a secure profile, that is, read–only access. AgentX allows interaction with third party subagents, provided that these subagents support AgentX over Unix Domain Sockets. For security reasons, AgentX is not supported over TCP/UDP. For more information on the AgentX protocol, see http://www.ietf.org/rfc/rfc2741.txt Edit the main /etc/sma/snmp/snmpd.conf configuration file to configure the System Management Agent to use the AgentX protocol. By default, the AgentX protocol is disabled. The following procedure describes how to enable the AgentX protocol. To enable the AgentX protocol add the following to /etc/sma/snmp/snmpd.conf file:

master agentx

restart the sma (system management agent).
# svcadm restart svc:/application/management/sma:default

Using the SMA
To view the status of the agent
# svcs svc:/application/management/sma:default

To See Which MIBs Are Initialized
# /usr/sfw/sbin/snmpd -Dregister_mib -Dmib_init -L

To Check the Disk Space and Network on a Local or Remote Machine
# /usr/sfw/bin/snmpwalk -v1 -c public localhost HOST-RESOURCES-MIB::hrStorageIndex

# /usr/sfw/bin/snmpget -v1 -c public localhost HOST-RESOURCES-MIB::hrStorageSize.1

# /usr/sfw/bin/snmpwalk -v1 -c public localhost HOST-RESOURCES-MIB::hrStorageUsed

# /usr/sfw/bin/snmpnetstat -v 2c -c public -a localhost

if all these works, congratulations, the sma is configured and brought on properly.

Tuesday, June 12, 2007

Common Array Manager

One of the most silly things Sun have done is to manage disk array using Java web console. When you want to set the disk array such as STK 6140 or 6540, you have to install Java, Tomcat and Common array manager on your Windows laptop.

When all components above installed succsessfully, remember to modify your firewall settings and registry, or you'll get "Authentication failed" message when logining to the Java web console.

using regedit on your XP system and set

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\ForceGuest

to 0.

Wednesday, May 30, 2007

Sun Java Communications Suite 5 on Multi-Hosts

This deployment example describes how to install Sun Java™ Communications Suite 5 software on multi-hosts for a functioning deployment. This document is intended for any evaluator, system administrator, or installation technician who wants to install and evaluate the services delivered by these components.

This deployment example contains the following sections:
  • Summary of Changes
  • About This Deployment Example
  • Installing the Example
  • Configuring Components
  • Starting and Stopping Communications Services
  • Accessing Sun Resources Online

Summary of Changes

The following describes the changes that have occurred from the Sun Java System Communications Services 6 2005Q4 single host installation example (as described in Deployment Example: Sun Java Communications Services 2005Q4 on a Single Host):

  1. Portal Server is no longer installed.
  2. Solaris™ Operating System (OS) 10 is used.
  3. Hosted domains are now configured.
  4. All Communications Express UI rendering elements are now deployed in the same web container to support Single Sign-On. As a result, Access Manager is no longer required for Communications Express and there are fewer configuration settings that you have to enter

About This Deployment Example

This section provides an overview of this deployment scenario, the hardware and software used, and the procedures you follow to install, configure, and use this deployment.

This section contains the following topics:

  • Installation Overview
  • Implementation Specifications for This Example
  • Summary of Installation and Configuration

Installation Overview

This scenario results in a functioning deployment suited for the evaluation and testing of Sun Java Communications Suite 5. These instructions are merely intended to guide the evaluator through an initial installation.

What Components Are Installed?

This example guides you through installing the following Communications Suite and Java Enterprise System components:

  • Sun Java System Messaging Server 6.3
  • Sun Java System Calendar Server 6.3
  • Sun Java System Communications Express 6.3
  • Sun Java System Directory Preparation Tool 6.4
  • Communications Services Delegated Administrator 6.4
  • Sun Java System Web Server 7.0
  • Sun Java System Message Queue 3.7 UR1
  • Sun Java System Access Manager 7.1
  • Sun Java System Directory Server Enterprise Edition 6.0
  • All Shared Components

The example installation in this document uses the following data that you change according to your installation and test machine:

  • domain name: abc.beta.com
  • host name: am(172.18.61.126), ds(172.18.61.125), mail(172.18.61.95) (all running Solaris OS 10 x86)
  • password: adminpass used for all passwords except the amldapuser password

Recommended Ports

For convenience, this example uses a set of recommended ports. If you use different ports, be sure to make the changes consistently throughout the installation. For example, this document uses port 8800 for the Web Server admin port. If you decide to use port 3333 instead, be sure to change it everywhere 8800 occurs in the deployment example.



Note

Unless otherwise indicated, all of the examples and instructions in this document assume that you are logged in as root.


Implementation Specifications for This Example

This section describes implementation specifications for this example.

Table 2 Evaluation Host Hardware and Software Requirements

Component

Platform Requirement

CPU

Intel

RAM

2 Gbytes or more

Disk space

2.2 Gbytes or more free disk space for installed software. 2.1 Gbytes additional disk space may be needed for temporary storage of Communications Suite zip files.

Software

Solaris 10 OS.

mail, am and ds are seprated zones on a single host.

Users

Provisioned with test users

Table 3 Client Software Requirements

Component

Platform Requirement

Browser

Netscape™ Communication 7.1, Internet Explorer 6.0 sp4+, or Mozilla™ 1.7.12, Firefox 2.0.3

Summary of Installation and Configuration

Installing and configuring this example involves the following high-level steps:

  1. Preparing the system for Sun Java Communications Suite
  2. Running the Java ES installer and selecting the necessary components
  3. Installing Messaging Server, Calendar Server, Communications Express, Delegated Administrator, Instant Messaging, Web Server, Directory Server, and Access Manager software
  4. Configuring Delegated Administrator
  5. Configuring Messaging Server
  6. Configuring Calendar Server
  7. Configuring Communications Express
  8. Configuring users

Installing the Example

This section describes how to install and configure components on multiple systems for evaluation purposes. Some components are configured after installation, using component configuration tools.

This section contains the following topics:

  • Checking Installation Requirements
  • Installing the Components
  • Uninstalling the Components

Checking Installation Requirements

Before you install components, use the steps in this section to make sure the computer on which you are installing is ready.

To Check System Requirements

The computer should meet the following requirements:

  • These instructions assume all actions are performed by the root user.
  • The computer must have 2.2 GB free disk space for the zip files and zip image. Recommendation: Have image already unzipped.

To Check DNS

Verify that DNS is running and configured properly:

  1. Make sure that the /etc/resolv.conf file has name server entries with the IP addresses of valid name servers. For example:

  2. domain abc.beta.com

    nameserver 172.18.61.120

  3. Make sure that the /etc/hosts file has an entry for the fully qualified host name of the server. This fully qualified host name should be listed before the non fully qualified host name. For example:

  4. 172.18.61.125 ds

    172.18.61.126 am

    172.18.61.95 mail

  5. Make sure that the /etc/nsswitch.conf file is configured to use files first to resolve host names. The hosts line in the nsswitch.conf file should list files first in its entry:
  6. hosts: files dns

To Unzip the Sun Java Communications Services Zip Files

  1. Download the Communications Suite software.
  2. Create a filesystem and mount it on the global zone, and mount it as lofs to non-global zones.
  3. Unzip the Communications Suite zip files in this directory. Then run the Java ES installer as described in the next section.

Installing the Components

You install Communications Suite components by running the Java Enterprise System installer.

To Start the Installer

  1. Log in as root to the machine on which you are installing Communications Suite software.
  2. Change to the JCS5 directory where you stored and unzipped the Communications Suite zip files.
  3. Change to the sun_java_commsuite5 directory
  4. Change to the Solaris_x8 platform directory.
  5. Start the Java Enterprise System installer in graphical mode.
  6. ./installer


    Note

    Solaris OS 10 installs an earlier version of the Message Queue (and possibly other software). You may need to upgrade this product prior to installing Communications Suite software. The Java ES installer detects that your system has an earlier version of a product and may present you with an option to Upgrade Existing Software. If you are presented with this option, continue with Step 8 that follows this note. If you are not presented with this option, skip to Step 19. If you are presented with the upgrade option, simply select the item to be upgraded and run the Java ES installer to upgrade the component. After the installer upgrades the component (or components), you can quit the installer then restart it to install the remaining Communications Suite components. Then you need to repeat the preceding steps.


  7. Click Next at the Welcome page.
  8. Accept the license.
  9. Select the Upgrade existing software radio button then click Next to continue.
  10. Select the following products then click Next:
    • Sun Java System Message Queue 3.7 UR1
    • All Shared Components

    • Note

      Also select the checkbox for multilingual packages if desired.



  11. Click Next to upgrade shared components.
  12. The installer verifies system requirements. When the installer finishes, click Next.
  13. The installer displays the Ready to Upgrade window.

  14. Click Next.
  15. Deselect the registration checkbox then click Install.
  16. The upgrade begins. When the upgrade finishes, the Installation Complete page appears.

  17. Click Close to exit the installer.
  18. Restart the Java Enterprise System installer in graphical mode.
  19. ./installer &

  20. Click Next at the Welcome page.
  21. Accept the license.
  22. The Choose to Upgrade window appears.

  23. Select the Install new software radio button then click Next to continue.
  24. Select the following products then click Next to Continue
    • on mail.abc.beta.com select the following components:
      • Sun Java System Messaging Server 6.3
      • Sun Java System Calendar Server 6.3
      • Sun Java System Message Queue 3.7 UR1
    • on am.abc.beta.com select the following components:
      • Sun Java System Communications Express 6.3
      • Communications Services Delegated Administrator
      • Sun Java System Web Server 7.0
      • Sun Java System Access Manager 7.1 (only the required software; no Client SDK)
    • on ds.abc.beta.com select the following components:
      • Sun Java System Directory Server Enterprise Edition 6.0 (Core, Control Center, Command-Line Utility)
      • Sun Java System Directory Preparation Tool 6.4
    • on all systems select:
      • All Shared Components



      Note

      Select the checkbox for multilingual packages if desired.



  25. Some versions of Solaris require that the Java SE SDK be upgraded prior to installation. If you are prompted to upgrade the Java SE SDK, choose the default selection to automatically upgrade the SDK to the version included with the installer. Then click Next to continue.
  26. Click Next to upgrade shared components.
  27. Specify the installation directories.
  28. Locations:

    • Directory Preparation Tool: /opt/SUNWcomds
    • Directory Server: /opt/SUNWdsee
    • Access Manager: /opt (will create /opt/SUNWam)
    • Web Server: /opt (will create /opt/SUNWwbsvr7)
    • Web Server Instance: /var/opt/SUNWwbsvr7
    • Messaging Server: /opt/SUNWmsgsr
    • Calendar Server: /opt (will create /opt/SUNWics5)
    • Delegated Administrator: /opt/SUNWcomm
    • Communications Express: /opt/SUNWuwc
  29. Accept the target directories for each product and click Next.
  30. The installer verifies system requirements, such as memory, disk space, and operating system patches. When done click Next.
  31. Choose Configure Now then click Next.
  32. The installer lists the components that must be configured after installation. This document informs you of the order in which to do so.
    • Sun Java System Messaging Server 6.3
    • Sun Java System Calendar Server 6
    • Sun Java System Delegated Administrator
    • Sun Java System Communications Express 6
  33. Click Next to configure the remaining components. You configure these components as part of the installation process.
  34. Continue with the following procedures to input the appropriate information for each of these products when prompted by the installer.



    Caution

    This example uses root and root (or root and other) for System User and System Group. These choices are satisfactory for evaluation deployments but not for production deployments. If you use this document to create a production deployment, use non-root identifiers. Refer to the Sun Java Enterprise System 2006Q4 Installation Guide for UNIX for more information.


To Select Common and Web Server Settings

This installation assumes the use of Sun Java System Web Server. You can choose an alternate web container but that is beyond the scope of this document.

  1. In the Account Preferences Panel, you are asked whether or not you want to use your UserID and Password information for all administrator accounts, or if you want to provide administrator account information for each component.
  2. Choose the All Administrator Accounts option and type admin for the Administrator User ID and adminpass for the Administrator Password then click Next.

  3. The installation script prompts for a series of options. The following table shows how to respond to the Common and Web Server configuration options.
  4. Table 4 Common and Web Server Settings

    Option

    [Default Value]

    Enter:

    Common Server Settings

    Host Name

    am

    accept default

    DNS Domain Name

    abc.beta.com

    accept default

    Host IP address

    172.18.61.126

    accept default

    System user

    root

    accept default

    System group

    root

    accept default

    Web Server Settings

    Type of configuration

    [Configure Administration Instance as Server]

    accept default

    Start servers after system startup

    [Unchecked]

    accept default

    Server Host

    am.abc.beta.com

    accept default

    Admin Server SSL port

    8989

    accept default

    Admin Server HTTP port

    8800

    accept default

    Admin runtime user ID

    root

    accept default

    Web Server Name

    am.abc.beta.com

    accept default

    HTTP port

    80

    accept default

    Web Server runtime UNIX user ID

    webservd

    webservd (Use root if deploying Portal Server.)

    Document root directory

    [/var/opt/SUNWwbsvr7/https-am.abc.beta.com/docs]

    accept default

To Select Directory Server Settings

  • The following table shows how to respond to the Directory Server configuration options.
  • Table 5 Directory Server Settings

    Option

    [Default Value]

    Enter:

    Create a Directory Instance?

    [Yes]

    accept default

    Instance Directory

    [/var/opt/SUNWdsee/dsins1]

    accept default

    Directory Instance Port

    [389]

    accept default

    Directory Instance SSL Port

    [636]

    accept default

    Directory Manager DN

    [cn=Directory Manager]

    accept default

    System User

    [root]

    accept default

    System Group

    [root] (Use other on Solaris OS 9.)

    accept default

    Directory Manager Password

    [adminpass]

    accept default

    Suffix

    [dc=abc,dc=beta,dc=com]

    o=beta.com

To Select Access Manager Settings

  • The following table shows how to respond to the Access Manager configuration options.
  • Table 6 Access Manager Settings

    Option

    [Default Value]

    Enter:

    Install type

    Legacy Mode (version 6.x style)

    accept default

    Admin User ID

    amadmin

    accept default

    Password

    --

    adminpass

    LDAP User ID

    amldapuser

    accept default

    LDAP Password

    --

    nadminpass

    Password Encryption Key

    87dfjkauefjkdafdadf...

    accept default

    Access Manager Web Container Options

    [Sun Java System Web Server]

    Choose Sun Java System Web Server

    Hostname

    [am.abc.beta.com]

    accept default

    Services Deployment URI

    [amserver]

    accept default

    Common Domain Deployment URI

    [amcommon]

    accept default

    Cookie Domain

    [.beta.com]

    .beta.com

    Password Deployment URI

    [ampassword]

    accept default

    Console Protocol

    [HTTP]

    accept default

    Administration Console

    [Deploy new console]

    accept default

    Console Deployment URI

    [amconsole]

    accept default

    Console Host Name

    [am.abc.beta.com]

    accept default

    Console Port

    [80]

    accept default

    Choose Directory Server

    ds.abc.beta.com:389

    accept default

    Directory Server provisioned with user data?

    [No]

    accept default

To Install the Software

  1. Click Install when you are satisfied with the Ready to Install list.
  2. Because of the number of components selected, the installation process can be lengthy.

  3. When installation is complete, the Installation Complete page is displayed. Click Close to exit the installer.

To Verify the Installation

Once the installation has completed, start LDAP and verify that it works.

  1. On ds.abc.beta.com, start Directory Server:
  2. cd /opt/SUNWdsee

    cd ds6/bin

    ./dsadm start /var/opt/SUNWdsee/dsins1

  3. On am.abc.beta.com, start Web Server:
  4. cd /var/opt/SUNWwbsvr7/admin-server/bin

    ./stopserv

    ./startserv

    cd /var/opt/SUNWwbsvr7/https-am..beta.com/bin

    ./stopserv

    ./startserv

    The Web Server can take a while to start.

  1. Verify that you can log in to the Access Manager console as amadmin. The URL for the Access Manager console is:
  2. http://fully-qualified-hostname:web-server-port/amconsole

    In this example, type:

    http://am.abc.beta.com/amconsole

    Use amadmin and adminpass as the user ID and password.

  3. After verifying that you can log in to the Access Manager console, log out.
  4. Proceed to Configuring Components for instructions on how to configure Communications Suite components.

Uninstalling the Components

After you complete your evaluation, you can use the Java Enterprise System uninstaller to remove the components that you installed. See the Sun Java Enterprise System 5 Installation Guide for UNIX:

Configuring Components

This section describes how to configure and start the components that you use in this example deployment.

This section contains the following topics:

  • Before You Begin
  • Preparing the Directory and Configuring Messaging Server
  • Configuring Delegated Administrator and Communications CLI
  • Configuring Messaging Server
  • Configuring Calendar Server
  • Configuring Communications Express
  • Verifying the Configuration
  • Creating Additional User Accounts and Groups

Before You Begin

Before you configure the Messaging Server software, you need to create the mailsrv user and disable the sendmail process. The mailsrv user is used in Configuring Messaging Server.

To Create the mailsrv User

  • Messaging Server requires a mail server user, which must exist before you can finish the installation. You can skip this step, because the group and user are created during the configuration time. However, you might want to create the mailsrv user yourself as shown below. For this installation, use the useradd command to create a user with username mailsrv and group mail. You use this ID in Step d (under Step 2).
  • Type the following commands to create the user mailsrv with a user ID of 7633, a group ID of mail, and a user home directory of /export/home/mailsrv on mail.abc.beta.com host. (This assumes the /export/home directory already exists.)


    groupadd mail

    useradd -u 7633 -g mail -d /export/home/mailsrv -m mailsrv

To Disable sendmail

  • Prior to configuring Messaging Server, use the following to kill the sendmail process and prevent it from starting during bootup.
  • For Solaris OS 10:


    svcadm disable sendmail

Preparing the Directory and Configuring Messaging Server

This section explains how to prepare the Directory Server LDAP schema and configure Messaging Server.

To Apply Schema 2 to Your Directory Tree

  1. Run the comm_dssetup.pl script on ds.abc.beta.com:
  2. cd /opt/SUNWcomds/sbin

    /usr/bin/perl comm_dssetup.pl

  3. Type y to continue.
  4. The perl script prompts for a series of options. The following table shows how to respond to the prompts.

    Table 7 Values for comm_dssetup.pl Script

    Option

    [Default Value]

    Enter:

    Directory Server Root

    [/var/opt/SUNWdsee/dsins1]

    /var/opt/SUNWdsee/dsins1

    Directory Manager DN

    [cn=Directory Manager]

    accept default

    Directory Manager Password

    --

    adminpass

    Use directory server for users/groups

    [Yes]

    accept default

    Users/Groups base suffix

    [o=beta.com]

    accept default

    Schema type?

    [2]

    accept default

    Update the schema files?

    [yes]

    accept default

    Configure new indexes?

    [yes]

    accept default

    Reindex new indexes?

    [yes]

    accept default

  5. Confirm your choices and type y to continue. The comm_dssetup script proceeds.
  6. When prompted, type y to continue with script.
  7. Continue with the next section after the comm_dssetup script finishes and displays its “Successful Completion” message.

Configuring Delegated Administrator and Communications CLI

This section describes configuring Delegated Administrator console and utility, which provide user management.

To Configure Delegated Administrator

  1. Run the configurator script on am.abc.beta.com:
  2. cd /opt/SUNWcomm/sbin

    ./config-commda

  3. Click Next in the Welcome window.
  4. Accept the default for the Directory to store User Mgmt data files: [/var/opt/SUNWcomm]
  5. If the directory does not exist, click Create Directory to create the directory.

  6. Install Delegated Administrator Utility, Console, and Server.
  7. The installation script prompts for a series of options. Use the following table to respond to the configuration options:

    Table 8 Values for config-commda Script

    Option

    [Default Value]

    Enter:

    AM Hostname

    [am.abc.beta.com]

    accept default

    AM Port

    [8080]

    80

    Default Domain

    [abc.beta.com]

    accept default

    Default SSL Port

    [443]

    accept default

    Web Container

    [Web Server 6.x]

    Web Server 7.x

    Install Root Directory (for DA Console)

    [/opt/SUNWwbsvr7]

    accept default

    Instance Root Directory

    [/var/opt/SUNWwbsvr7]

    accept default

    Server Instance Identifier

    [am.abc.beta.com]

    accept default

    Virtual Server Identifier

    [am.abc.beta.com]

    accept default

    Web Server HTTP Port

    [80]

    accept default

    Administration Server Port

    [8800]

    accept default

    Administrator User ID

    [admin]

    accept default

    Administrator Password

    --

    adminpass

    Secure Administration Server Instance?

    [Unchecked]

    accept default

    Default Domain Separator

    [@]

    accept default

    Access Manager Base Directory

    [/opt/SUNWam]

    accept default

    Install Root Directory (for DA Server)

    [/opt/SUNWwbsvr7]

    accept default

    Instance Root Directory

    [/var/opt/SUNWwbsvr7]

    accept default

    Server Instance Identifier

    [am.abc.beta.com]

    accept default

    Virtual Server Identifier

    [am.abc.beta.com]

    accept default

    Web Server HTTP Port

    [80]

    accept default

    Administration Server Port

    [8800]

    accept default

    Administrator User ID

    [admin]

    accept default

    Administrator Password

    --

    adminpass

    Secure Administration Server Instance?

    [Unchecked]

    accept default

    URL of Directory Server

    [ldap://ds.abc.beta.com:389]

    accept default

    Bind As

    [cn=Directory Manager]

    accept default

    Password

    --

    adminpass

    AM Top level admin

    [amadmin]

    accept default

    AM admin password

    --

    adminpass

    Access Manager Internal LDAP Auth Username

    amldapuser

    accept default

    AM Internal LDAP Auth Password for amldapuser

    --

    nadminpass

    Organization DN

    [o=abc.beta.com,o=beta.com]

    accept default

    Top Level Admin for Default Organization

    [admin]

    accept default

    Password

    --

    adminpass

    Load Sample Service Packages

    --

    Yes (Checked)

    Load Sample Organizations

    --

    Yes (Checked)

    Preferred Mailhost for Sample

    [mail.abc.beta.com]

    accept default

  8. Select Configure Now.
  9. The script begins to run.

  10. When the panel displays “All Tasks Passed,” click Next to continue.
  11. Two warnings appear: one is remind you to restart Web Server; the other is to remind you to enable the mail and calendar services in the domain. The next steps correct these problems.

  12. Click Close to complete the configuration.
  13. Restart Web Server:
  14. cd /var/opt/SUNWwbsvr7/https-am.abc.beta.com/bin

    ./stopserv

    ./startserv

  15. Modify the mail and calendar domains, and create users by using the commadmin utility:

  16. /opt/SUNWcomm/bin/commadmin domain modify -D admin -w adminpass -X am.abc.beta.com -n abc.beta.com -p 80 -d abc.beta.com -S mail,cal -H mail.abc.beta.com

    /opt/SUNWcomm/bin/commadmin user create -D admin -F Alvin -l alvin -L Ding -n abc.beta.com -p 80 -w adminpass -W demo -X am.abc.beta.com -S mail,cal -E alvin@abc.beta.com -H mail.abc.beta.com

    /opt/SUNWcomm/bin/commadmin user create -D admin -F Calendar -l calmaster -L Master -n abc.beta.com -p 80 -w adminpass -W adminpass -X am.abc.beta.com -S mail,cal -E calmaster@abc.beta.com -H mail.abc.beta.com

    Create as many users as you need. Steps later in this document show how to add Presence and Instant Messaging services to those users.

Configuring Messaging Server

This section describes configuring Messaging Server, including configuring the Webmail port.

To Configure Messaging Server

  1. Run the Messaging Server configure script on mail.abc.beta.com:
  2. cd /opt/SUNWmsgsr/sbin

    ./configure

    The Configuration Wizard appears. Read the introductory information and proceed by clicking Next.

  3. Verify the following:
    1. Fully qualified host name of Messaging Server, FQHN: [mail.abc.beta.com]
    2. Directory to store config/data files: [/var/opt/SUNWmsgsr]
    3. When prompted, choose to create the new directory.

    4. Install MTA, Message Store, and Webmail Server. There is no need to install the Multiplexor for this deployment.
    5. Name of the mail server Unix user: Unix username [mailsrv]
    6. Unix group: [mail]
  4. The installation script prompts for a series of options. Use the following table to respond to the configuration options:
  5. Table 9 Values for Messaging Server configure Script

    Option

    [Default Value]

    Enter:

    User/Group Server LDAP

    [ldap://ds.abc.beta.com:389]

    accept default

    Bind As

    [cn=Directory Manager]

    accept default

    Password

    --

    adminpass

    Postmaster email address

    --

    alvin@abc.beta.com

    Password for Messaging Server accounts

    --

    adminpass

    Default email Domain

    [abc.beta.com]

    accept default

    Organization DN

    [o=abc.beta.com,o=beta.com]

    accept default

  6. Click Next, then click Configure Now.
  7. You will receive an error about the Webmail port being in use. Click OK to acknowledge this message. The configuration begins. The following step corrects the Webmail port problem.

  8. When the configuration is finished, click Next to continue, then click Close to exit.


  9. Note

    You might observe a failure such that not all the messaging configuration tasks have passed. If this is the case, rerun the Messaging Server configuration script by repeating all the steps above. When prompted for the directory to store configuration or data files, use the same directory as before. A popup window will state that this directory already exists. Click Accept Anyway to verify that the same directory will be used.



  10. (Option)Configure the Webmail port:
  11. /opt/SUNWmsgsr/sbin/configutil -o service.http.port -v 8080

  12. Start Messaging Server:
  13. /opt/SUNWmsgsr/sbin/stop-msg

    /opt/SUNWmsgsr/sbin/start-msg

  14. Continue with the next section to configure Calendar Server.

Configuring Calendar Server

This section shows you how to run the Calendar Server configuration script and how to verify the Calendar Server configuration.

To Configure Calendar Server

  1. Run the Calendar Server configurator script on mail.abc.beta.com:
  2. cd /opt/SUNWics5/cal/sbin

    ./csconfigurator.sh

  3. Click Next at the Welcome page.
  4. The installation script prompts for a series of options. Use the following table to respond to the configuration options:
  5. Table 10 Values for Calendar Server csconfigure Script

    Option

    [Default Value]

    Enter:

    LDAP Server Host Name

    [ds.abc.beta.com}

    accept default

    LDAP Server Port

    [389]

    accept default

    Directory Manager DN

    [cn=Directory Manager]

    accept default

    Directory Manager Password

    --

    adminpass

    Enable Virtual Domains support

    [Checked]

    accept default

    Virtual Domains setting: Default domain

    [abc.beta.com]

    accept default

    Calendar Administrator Username

    [calmaster]

    accept default

    Calendar Administrator Password

    --

    adminpass

    Calendar Administrator Email address

    [calmaster@abc.beta.com]

    accept default

    Site administrator

    [Yes]

    accept default

    Set up a Front End/Back End deployment

    [Unchecked]

    accept default

    Email Alarms

    [Enabled]

    accept default

    Administrator Email Address

    [calmaster@abc.beta.com]

    accept default

    SMTP Host Name

    [mail.abc.beta.com]

    accept default

    Service Port

    [80]

    3080

    Maximum Sessions

    [5000]

    accept default

    Maximum Threads

    [20]

    accept default

    Number of server processes

    [1]

    accept default

    Runtime User ID

    [icsuser]

    accept default

    Runtime Group ID

    [icsgroup]

    accept default

    Start Calendar Server after Successful Configuration

    [Unchecked]

    accept default

    Start Calendar Serve on System Startup

    [Checked]

    Unchecked

    Configuration Directory

    [/etc/opt/SUNWics5/config]

    accept default

    Database Directory

    [/var/opt/SUNWics5/csdb]

    accept default

    Attachment Store Directory

    [/var/opt/SUNWics5/astore]

    accept default

    Logs Directory

    [/var/opt/SUNWics5/logs]

    accept default

    Temporary Files Directory

    [/var/opt/SUNWics5/tmp]

    accept default

    Enable Archive

    [Checked]

    accept default

    Archive Directory

    [/var/opt/SUNWics5/csdb/archive]

    accept default

    Enable Hot Backup

    [Checked]

    accept default

    Hot Backup Directory

    [/var/opt/SUNWics5/csdb/hotbackup]

    accept default

    Keep archives for (in days)

    [Minimum: 3, Maximum: 6]

    accept default

    Keep hot backups for (in days)

    [Minimum: 3, Maximum: 6]

    accept default

    Same as archive checkbox

    [Checked]

    accept default

  6. Click the Configure Now button to configure Calendar Server.
  7. The Configuration begins and takes less than two minutes.

  8. Click Next when the configuration is complete.
  9. Click the Details button to verify that all packages configured correctly. Click the Close button to exit the configurator.
  10. Start the Calendar Server daemons:
  11. cd /opt/SUNWics5/cal/sbin

    ./stop-cal

    ./start-cal

  12. Continue with the next section to configure Communications Express.

Configuring Communications Express

This section shows you how to run the Communications Express configuration script.

To Configure Communications Express

  1. Run the Communications Express configurator script on am.abc.beta.com:
  2. cd /opt/SUNWuwc/sbin

    ./config-uwc

  3. Click Next at the Welcome page.
  4. Accept the default for Directory to store configuration and data files: [/var/opt/SUNWuwc]
  5. Choose to create the directory when prompted.

  6. Install the Mail and Calendar Components.
  7. The installation script prompts for a series of options. Use the following table to respond to the configuration options:

    Table 11 Values for config-uwc Script

    Option

    [Default Value]

    Enter:

    Hostname

    [am]

    accept default

    DNS Domain

    [abc.beta.com]

    accept default

    Web Container

    [Web Server]

    accept default

    Web Server Root Directory

    [/opt/SUNWwbsvr7]

    accept default

    Web Server Config Root Directory

    [/var/opt/SUNWwbsvr7]

    accept default

    Web Server Instance Identifier

    [am.abc.beta.com]

    accept default

    Virtual Server Identifier

    [am.abc.beta.com]

    accept default

    HTTP Port

    [80]

    accept default

    Administration Server Port

    [8800]

    accept default

    Administrator User ID

    [admin]

    accept default

    Administrator Password

    --

    adminpass

    Secure Administration Server Instance

    [Unchecked]

    accept default

    Web Container User ID

    [webservd]

    accept default

    Web Container Group IP

    [webservd]

    accept default

    URI Path

    [/uwc]


    /mail

    Enable Hosted Domain Support

    [Unchecked]

    Checked (Yes)

    URL of Directory Server

    [ldap://wireless.comms.beta.com:389/]

    ldap://ds.abc.beta.com:389/

    Bind DN

    [cn=Directory Manager]

    accept default

    Password

    --

    adminpass

    DC Tree suffix

    [o=beta.com]

    accept default

    Default Domain

    [abc.beta.com]

    accept default

    Enable Access Manager for Single Sign-On

    [Unchecked]

    accept default (not enabled)

    Webmail Server Host Name

    [wireless.comms.beta.com]

    mail.abc.beta.com

    Webmail Server Port Number

    --

    80

    Enable login in secure mode

    [Unchecked]

    accept default (not secure)

    Webmail Server SSL Port Number

    --

    Leave blank

    Webmail Admin User ID

    --

    admin

    Admin Password

    --

    adminpass

    Calendar Server Hostname

    [wireless.comms.beta.com]

    mail.abc.beta.com

    Calendar Server Port

    [9004]

    3080

    Calendar Admin User ID

    [calmaster]

    accept default

    Calendar Administrator User Password

    --

    adminpass

    URL of PAB Directory Server

    [ldap://ds.abc.beta.com:389]

    accept default

    Bind As

    [cn=Directory Manager]

    accept default

    Password

    --

    adminpass

  8. Click the Configure Now button to configure Communications Express.
  9. After the configuration is finished, click Next to continue.
  10. The system displays a message to restart the Web Server instance.

  11. Click OK.
  12. Click Next to continue.
  13. Click Close to quit the Communications Express configurator script.
  14. Restart Web Server:
  15. cd /var/opt/SUNWwbsvr7/https-am.abc.beta.com/bin

    ./stopserv

    ./startserv

Verifying the Configuration

This section describes how to verify the configuration for Communications Express client, Delegated Administrator, and Instant Messenger. You simply verify that you can log in to the various services. After you are satisfied that you can log in, log out.

  1. Enable Communications Express mail (Webmail) on mail.abc.beta.com.

    cd /opt/SUNWmsgsr/sbin

    ./configutil -o local.webmail.sso.uwcenabled -v 1

    ./stop-msg

    ./start-msg

  1. In your web browser, open the following URL for the Communications Express client:
  2. http://am.abc.beta.com/mail

  3. Log in as one of the users created earlier.
  4. If you created a user according to the previous steps, type alvin for the user name, and demo for the password.

  5. In your web browser, open the following URL for Delegated Administrator:
  6. http://am.abc.beta.com/da/DA/Login

    Log in as user admin. The password is adminpass.

  7. Continue with the next section to configure Access Manager.

Creating Additional User Accounts and Groups

This section describes how to create additional users accounts and groups. Users and groups created in this fashion work with both Communications Services products and Portal Server (if you add Portal Server to this deployment at a later date).

To Create End User Accounts and Groups

  1. The following example shows how to create users using the Delegated Administrator command-line utility and also how to create a group consisting of the users created.

  2. /opt/SUNWcomm/bin/commadmin user create -D admin -F Demo -l demo1 -L One -n abc.beta.com -p 80 -w adminpass -W demo -X am.abc.beta.com -S mail,cal -E demo1@abc.beta.com -H mail.abc.beta.com

    /opt/SUNWcomm/bin/commadmin user create -D admin -F Demo -l demo2 -L Two -n abc.beta.com -p 80 -w adminpass -W demo -X am.abc.beta.com -S mail,cal -E demo2@abc.beta.com -H am.abc.beta.com

    /opt/SUNWcomm/bin/commadmin group create -D admin -G Demostaff -n abc.beta.com -p 80 -w adminpass -X am.abc.beta.com -S mail -E Demostaff@abc.beta.com -H mail.abc.beta.com -o calmaster -m demo1 -m demo2



    Note

    Creating groups using the Delegated Administrator command-line interface will guarantee that these groups will be recognized by Access Manager as LDAP groups that can be searched on using Instant Messenger. They can also be used to send email to the entire group.


To Create User Accounts by Using Delegated Administrator Console

An alternative to using the Delegated Administrator command-line utility to create users is to use the Delegated Administrator console. The high-level steps to create users with Delegated Administrator console are the following:

  • Creating the user
  • Using Access Manager to add Services to those user accounts (if necessary)

Prior to using Delegated Administrator, check the resource.properties file to make sure users will be created using options consistent with your usage. Because this example uses a hosted domain scenario, you do not need to edit this file to make changes. If you do want to make changes, perform the following steps.

  1. Change directories to /var/opt/SUNWcomm/WEB-INF/classes/sun/comm/cli/server/servlet and edit the resource.properties file on am.abc.beta.com.
  2. The default settings for servicepackage refresh rates are as follows:

    servicepackage-ttlhours=5

    servicepackage-ttlminutes=0

    These settings establish the time between refreshes of service packages. In a demo scenario, service packages do not change very often so there is no need to refresh them every five minutes. Instead, they are refreshed every five hours. This improves Delegated Administrator performance. If you later decide to change the refresh rate, for example, to 20 hours, then make this change:

    servicepackage-ttlhours=20

    The default setting for calendar ID type is set using the switch-caltype variable as follows:

    switch-caltype=hosted

    The caltype can either be hosted or legacy. When you set caltype to hosted, users are created as uid@domain rather than uid, when you use Delegated Administrator to create them.

  1. If you make changes to the resource.properties file, restart Web Server:

    cd /var/opt/SUNWwbsvr7/https-am.abc.beta.com/bin

    ./stopserv

    ./startserv

  1. The first time you use Delegated Administrator console, you must add service packs to the organization that contains your users. In a web browser, open the URL for the Delegated Administrator console and log in as amadmin (password is adminpass):
  2. http://am.abc.beta.com/da/DA/Login

  3. Click the check box next to the comms.beta.com organization, then click the Allocate Service Package button.
  4. The Allocate Service Package window appears. Service packages are organized within a set of three pages.

  5. Click the Single Page button (to the right of the Compare button) to map the three pages into a single page view.
  6. Select the desired service packages.
  7. Choose from among earth, mars, mercury, and venus, because they have mail and calendar services enabled. Each service package has a different mail quota.

  8. Click Next to continue.
  9. Accept the Mail Service defaults, then click Next.
  10. If you chose a service package that contains Calendar, type the Calendar Host wireless.comms.beta.com and Anonymous Login Yes. Then click Next.
  11. Select the quantity of each service pack desired then click Next.
  12. A quantity of 100 is adequate for demo purposes.

  13. Review your choices and click Finish.
  14. The service packages are created.

  15. Once the service packs have been allocated for the domain, you can now add users to the domain. Click the domain link, comms.beta.com.
  16. Click the New button to create a new user account within this domain.
  17. Type a First Name (such as Demo), Last Name (such as Five), Display Name (such as Demo Five), then click Next.
  18. There is no need to type postal information, so click Next.
  19. Select a service pack for this user, for example, earth, then click Next.
  20. Type the email address, for example, demo5@comms.beta.com. Mail Delivery Option should be Local Inbox. Leave the other fields blank.
  21. Click Next.
  22. Choose your calendar preferences and pick your preferred timezone. Then click Next.
  23. Type Login ID and Password. In this example, Login ID is demo5 and password is demo.
  24. Click Next to continue.
  25. Verify the user information then click Finish to create the user account.
  26. Log in to Access Manager as amadmin and verify that the IM and Presence services have been added to the newly created user. If not, assign services to those users as described previously in To Add Services to the Lower Level Organization.

Starting and Stopping Communications Services

This section describes the commands needed to start and stop all the Communications Services.

To Start and Stop Services

You should stop services before attempting to start them.

  1. To stop all services:
  2. Table 12 Commands to Stop Communications Services

    Service

    Command

    Instant Messaging

    /opt/SUNWiim/sbin/imadmin stop

    Access Manager

    /etc/init.d/amserver stop

    Web Server

    /var/opt/SUNWwbsvr7/https-wireless.comms.beta.com/bin/stopserv

    Web Server Administrator

    /var/opt/SUNWwbsvr7/admin-server/stopserv

    Calendar Server

    /opt/SUNWics5/cal/sbin/stop-cal

    Messaging Server

    /opt/SUNWmsgsr/sbin/stop-msg

    Directory Server

    /opt/SUNWdsee/ds6/bin/dsadm stop /var/opt/SUNWdsee/dsins1

  3. To start all services:
  4. Table 13 Commands to Start Communications Services

    Service

    Command

    Directory Server

    /opt/SUNWdsee/ds6/bin/dsadm start /var/opt/SUNWdsee/dsins1

    Messaging Server

    /opt/SUNWmsgsr/sbin/start-msg

    Calendar Server

    /opt/SUNWics5/cal/sbin/start-cal

    Web Server Administrator

    /var/opt/SUNWwbsvr7/admin-server/startserv

    Web Server

    /var/opt/SUNWwbsvr7/https-wireless.comms.beta.com/bin/startserv

    Access Manager

    /etc/init.d/amserver start

    Instant Messaging

    /opt/SUNWiim/sbin/imadmin start

Accessing Sun Resources Online

The docs.sun.comSM web site enables you to access Sun technical documentation online. You can browse the docs.sun.com archive or search for a specific book title or subject. Books are available as online files in PDF and HTML formats. Both formats are readable by assistive technologies for users with disabilities.

To access the following Sun resources, go to http://www.sun.com:

  • Downloads of Sun products
  • Services and solutions
  • Support (including patches and updates)
  • Training
  • Research
  • Communities (for example, Sun Developer Network)


Fast, Safe, Open, Free!
Open for business. Open for me! » Learn More