Skip to end of metadata
Go to start of metadata

Overview

XSEDE invites researchers to submit Research Allocation Requests 4 times each year.  (Trial/Campus Champions/Startups/Education Allocation Requests can be submitted anytime.)  Requests may be submitted the quarter before they are needed during designated submission window as listed in Table 1 below.  Allocation requests are only accepted electronically and must be made through the XRAS system via the XSEDE User Portal (XUP). Requests include annual submissions to initiate or continue allocations as well as manage allocations during an allocation period.

XSEDE Resource Allocation Committee (XRAC) members meet quarterly to review requests based on merit and then award available SUs.  Requests are assigned to reviewers by matching the field of sciences (FOS) of the Reviewer with those selected by the request’s PI, and relying on Title and Keyword Preferences (reviewers rank their expertise preference for each request title/keywords set in their FOS).  

 

Submission Period

Allocation Begin Date

Dec 15  -- Jan 15

Apr 1

Mar 15 – Apr 15

July 1

Jun 15 – Jul 15

Oct 1

Sept 15 – Oct 15

Jan 1

Table 1. Research Allocation Deadlines        

Review Criteria

The Main Document of the resource request will be reviewed against three criteria, which apply across all types of resources, with the level of detail of the review rising with the size of the requested resources:

  1. Appropriateness of Methodology: For compute requests, the choice of applications, methods, algorithms and techniques to be employed to accomplish the stated scientific objectives should be reasonably described and motivated. For storage requests, the data usage, access methods, algorithms and techniques to be employed to accomplish the stated research objectives should be reasonably described and motivated. For shared collections, requestors should describe the public or community access methods to be provided.
  2. Appropriateness of Research Plan: The steps in the research plan should explain how the research objectives would be achieved. For compute resources, the proposed computations should encompass simulation parameters (step size, time scale, ensemble parameters, etc.) that are needed to obtain accurate and meaningful results, as well as the human resources that can be devoted to the task. The amount of resources requested should be derived from the methodology and the research plan. If there are serious concerns about the research plan, reviewers will document those concerns in their reviews, and the PI may choose to address those concerns in an Appeal.
  3. Efficient Use of Resources: The resources selected should be used as efficiently as is reasonably possible and in accordance with the recommended use guidelines of those resources. For computational resources, performance and parallel scaling data should be provided along with a discussion of optimization and/or parallelization work to be done to improve the applications. If the reviewers conclude that the request is more appropriate on XSEDE resources other than those requested, they may recommend an allocation on those other resources instead.

In considering these criteria, the XRAC recognizes that scientific productivity is the end goal. If adapting to less familiar but, in the view of the panel, better architectures or algorithms requires a significant learning curve for the proposer, with a concomitant interruption of scientific productivity, the XRAC may suggest the alternatives, but nevertheless grant the requested resources with the proposed architecture and methods. In exceptional cases, where the reviewers conclude "beyond a reasonable doubt" that the proposed methods are so inefficient that they amount to a waste of public resources, they should not approve the request until their concerns are addressed by the proposer. Reviewers are encouraged to recommend extended collaborative support, even for those proposers who have not requested it, as part of their recommendations for addressing shortcomings. For storage resources, information on required performance and expected access patterns should be provided for all data and collections to be stored and used along with a discussion of work done or planned to improve the efficiency of the data use.

Note that the availability of resources is not considered in the review. Only the merit of the proposal is. Necessary reductions due to insufficient resources will be made after the merit review, under NSF guidelines, as described in Section 6.4.1.

These are used in all types of XRAC research requests (New/Renewal/Supplements/Appeals), and form the basis for assigning a Suggested Allocation.  All PIs of a request (PI/Co-PIs) are instructed that they must include information about the grant(s) that support(s) their Computational Research Plan.  The funding totals, funding sources, and “the percentage of the annual funding for the grant(s) that will be associated with the activities to be supported by XSEDE resources” must be provided, along with a few other details.  Reviewers use this information only to assess whether the PI has enough support to accomplish the Computation Research Plan, analyze data, prepare publications, etc.  This reviewer assessment is blind to the type or source of the funding.   If the funding is from a previously (nationally) peer-reviewed proposal, the proposed science is not re-reviewed in the XRAC process.  However, when this is not the case, or funding information is not supplied, the proposed science is reviewed by the XRAC reviewers. 

All renewal requests must provide a Progress Report (as a separate document.)  Reviewers evaluate the accomplishments of the previous Computational Research Plan, usage of the allocated resources, and scientific reports and publications.  An unexcused history of uncompleted allocation usage will often lead reviewers to award allocations lower than those requested. 

Recommended Allocation

At the quarterly XRAC meeting, reviewers convene to determine a Recommended Allocation for each XRAC Research request (and large Supplements or Justifications).  A preliminary discussion is held among each request’s reviewers, to derive a consensus Recommended Allocation from the set of individual Suggested Allocations.   In subsequent full-panel discussions, each review team presents a summary of the requests and their reviews, and the consensus Recommended Allocation. All members of the Panel Discussion (including attending site representatives) have an electronic copy of abstracts, reviews, requested amounts and reviewer names.  Wireless access at the meetings insures that reviewers can view the present requests and historical information. After the team’s presentation, the floor is open to discussion, and the Recommended Allocation is approved or adjusted by the XRAC members (site representatives have no vote and attend only in an advisory capacity).   Any special considerations are noted; these may include mobility of projects from one resource to another, panel discussion decisions, special needs and reasons for any exception from allocation Reconciliation.  At the end of the full Panel Discussion, the chair asks the committee members to propose any further review of requests that need re-evaluation in retrospect of the general disposition of all other awards.  (This is called a request for “normalization”.) 

In this full process, reviewers are asked to only review requests on their merits.  The reviewers are not informed about the total requests on the different systems until after their individual reviews have been completed and they arrive at the meeting, so that their initial review is based purely on the merit of the proposal, and is not relative to availability of allocation on an individual resource.  After their reviews are submitted, the allocation request total and available SUs for each resource are provided to reviewers before the Caucus to allow them time to prepare “mobility” information for requests which might need to be moved to alternate systems in a potential oversubscription situation.

The total of the Recommended Allocations is reported to NSF, so that NSF will be aware of the peer-review panel’s statement of the actual need for computations resources.

Allocations Awards

In advance of each XRAC meeting, site representatives determine the amount of time (SUs), storage space (TB) or other resources that can be provided at the quarterly meeting to satisfy allocation requests. These estimates are called the Available Allocations for each of their resources.  In general, approximately one quarter of the yearly available allocation for each system is made available at each quarterly meeting.  Sites make small adjustments to insure that batch queuing systems operate with reasonably low queue-wait times. (Cores x 8760 SUs/core-year x 0.80 x 1yr/4quarter = approximate Available Allocation per quarter.  The factor 0.8 is typically used to account for discretionary use, down time, scheduler efficiency etc.; but this may vary from system to system and from quarter to quarter.)  Summed over all systems, this is the total Available Allocation.

I. When the total Recommended Allocation is less than the total Available Allocation, the Recommended Allocations generally become the Awarded Allocations, though individual allocations may need to be moved to alleviate oversubscription on individual resources.

II. When the total Recommended Allocation exceeds the total Available Allocation, a condition of general oversubscription exists.  The remainder of this document deals with this case.

Reconciliation (Adjustments to Recommended Allocations to Eliminate Oversubscription)

After the Panel Discussion of the XRAC meeting, the total Recommended Allocation is determined and compared to the total Available Allocation across all resources. Transfers of allocations may be made for projects that are more suitable for execution on other resources; transfers may also be made for projects that can take advantage of other resources, hence balancing the load. When the total Recommended considerably exceeds Available Allocations a reconciliation process adjusts all Recommended Allocations to remove oversubscription. This adjustment process reduces large allocations more than small ones and gives preference to NSF-funded projects or project portions. Under the direction of NSF, additional adjustments may be made to achieve a balanced portfolio of awards to diverse communities, geographic areas, and scientific domains.

When the total Recommended Allocation exceeds the total Available Allocation and the ratio is near unity, the committee members are asked to reconsider requests for adjustments.  Adjustments are made to the Recommended Allocations to arrive at the Awarded Allocations. 

When the ratio of the total Recommended Allocation to total Available Allocation is far from unity, then there is a “Reconciliation” session that uniformly adjusts all Recommended Allocations to remove oversubscription (i.e. all Recommended Allocations are reduced until the total is equal to the total Available Allocation). This adjustment is not performed by the XRAC committee.  The XSEDE Allocations officers and site representatives use a formula and procedure approved by the the NSF program manager to make reconciliation adjustments to the Recommended Allocations to arrive at the Awarded Allocations. 

The adjustment for oversubscription is applied globally prior to moving awards from oversubscribed to undersubscribed resources; this avoids preferentially favoring those requests that targeted under-requested systems. But, systems that do not provide general-purpose batch computing (e.g., Condor Pools, VM hosting systems, GPGPU systems, visualization systems) will be excluded or considered as a separate adjustment group (further details below).

Requests are reduced by a global fraction. Additional adjustments are made for funding and size (the number of SUs requested.)  Any exceptions recommended by the committee are dealt with separately and on a case-by-case basis.  The details of how the reductions are done are listed below, but be warned, this is not simple to understand.

Adjustment Details

There are three factors that are included in the adjustment: A global adjustment factor (X) to reduce all Recommended Allocations; a factor (y) that gives NSF funded requests a priority (where mixed sources of funds are reduced in proportion to their NSF and non-NSF funding ratios); and a scaling factor (Z) that linearly reduces large requests more than small requests. These factors are listed in Table 3.

Adjustment Targets

Factor

Comments

Global, across the board

X

Set by oversubscription

Funding Source

Y

Preference to NSF-Funded research

Size of Award

Z

Small awards reduced less

Table 2. Factors used adjust Recommended Allocations for oversubscription.

 

The formula for adjusting an allocation for oversubscription is:

Alloc = [ (1-X)*Rn + (1-X*Y) * Ro ] Size(R,Z)            {for small oversubscription X<1/3}

Alloc = [ (1-x)*Rn + (1-X)*Y’ * Ro ] Size(R,Z)           {for large oversubscription X>=1/3}

 

R   = Recommended Allocation

Rn = Component of Recommended allocation (R) that is supported by NSF funding.

Ro = Component of Recommended allocation (R) that is supported by non-NSF funding.

X    = global scaling factor, between 0-1.

Y    = funding priority factor 2  (for large oversubscription)

Y’  = funding priority factor  0.5 (for high oversubscription)

Z   = size scaling factor 0.1

In practice, the funding and size factors (Y and Z) are set, and the global X factor is adjusted by non-linear Optimization (using the Microsoft Excel Solver plug-in). The XSEDE Allocations Coordinator will make appropriate adjustments for specific systems and use NUs when appropriate.   For instance, since the size of the requests for SMP machines are two orders less than those for the AQSL, Ranger and Kraken systems and are application specific, they will be considered separately.  Likewise, when the oversubscription is specific to particular systems (e.g. AQSL, Ranger and Kraken) the coordinator may limit the adjustment to these specific systems.  When it is necessary to eliminate oversubscription, the XSEDE Allocations Coordinator will include information about the reductions in Recommended Allocations in the award notifications. 

The final step reduces each request’s individual system allocation to achieve the targeted allocation total for that request, as well as balancing allocations between over-requested and under-requested systems. For example, a request for SUs on Ranger and Kraken many receive a different fraction on each machine to accommodate the difference in oversubscription on the two systems. Likewise, a Recommended Allocation on an over-requested system may be moved to an under-requested system. 

Oversubscription on “specialty” resources, such as the current SMP systems (Cobalt and Pople), will be handled as a separate adjustment set of systems, since recommended allocations on these systems  cannot be moved, due to large-memory/SMP requirements.

This final step requires a more heuristic approach, using information about the request’s “mobility” acquired from the XRAC during the Panel Discussions, alternate systems previously used or currently requested by the researchers, and special needs (data storage/visualization) that preclude splitting or moving a request.

The final step only deals with evening out the reduced Recommended Allocation across systems. Through the prior application of the adjustment formula the RPs can be assured that all reduced recommendations will fit within the available SUs, and that all PI requests are treated equitably when oversubscription exists for a common genre of XSEDE resources.

  • No labels