C USACE Model Certification Review

Model evaluation is the process for ensuring that numerical tools are scientifically defensible and transparently developed. Evaluation is often referred to as verification or validation, but it in fact includes a family of methods ranging from peer review to model testing to error checking (Schmolke et al. 2010). The USACE has established an ecological model certification process to ensure that planning models are sound and functional. These generally consist of evaluating tools relative to the three following categories: system quality, technical quality, and usability (EC 1105-2-412).

The NYBEM underwent USACE review and certification from April through August 2022. An April 2022 version of the models were reviewed by four reviewers representing coastal ecology (two USACE New England District biologists, “Reviewer-1” and “Reviewer-2”), an experienced R-programmer (USACE Environmental Laboratory, “Reviewer-3”), and a modeling policy expert (USACE Ecosystem Restoration Planning Center of Expertise, “Reviewer-4”). All comments follow the four-part comment structure of: (1) identify the problem, (2) describe the technical basis for the comment, (3) rate the significance or impact of the problem, and (4) recommend a mechanism for resolution. Comments and author responses are provided for transparency in review processes and long-term archival.

C.1 Reviewer-1

Comment 1.1: Section 3.2 - The Coastal and Marine Ecological Classification Standard (CMECS) was endorsed by the Federal Geographic Data Committee (FGDC) in 2012. As an FGDC standard, federally funded projects working with environmental data in coastal and marine settings should attempt to use CMECS as their primary classification system or attempt to crosswalk their system.

Basis: Because USACE is a federal agency and because this project is federally funded, the NYBEM conceptual model should classify ecosystem and habitat types in accordance with CMECS to promote the standardization of habitat/ecosystem classification. Furthermore, CMECS is decades more contemporary than Cowardin et al. (1979) and USFWS (1997). However, I understand that NYBEM is, in part, built off of other models that use different classification systems, so adopting CMECS may be difficult. Furthermore, because the goal of NYBEM is not necessarily to classify habitats but to assess the effects of hurricane barrier projects, a case can be made that it isn’t necessary to adopt CMECS. However, if the long-term goal is to develop this model into something that can be used in a broader capacity (i.e., in other regions for different coastal projects), then an attempt should be made to incorporate CMECS.
Significance: Medium.
Recommended Resolution: Least effort resolution is to include a statement as to why the current classification standard was chosen and a nod to why CMECS was not used as the primary classification standard, given the call for federal agencies to use CMECS.
Author Response: Non-concur. CMECS classifies ecosystems based on two major factors: (1) the aquatic setting and (2) the biogeographic setting. The aquatic setting is defined relative to salinity, depth, and tidal zones, which largely align with the NYBEM zonation scheme. The biogeographic setting is defined by regional setting, and a series of increasingly detailed filters related to water column, geoform, substrate, and biotic components. These detailed data were unavailable at the regional scale to map these features for the existing condition as well as forecast shifts in zonation through time. However, the model scope generally defines a biogeographic region (i.e., the New York Bight), and other biogeographic components were incorporated where data supported their use (e.g., a differentiation between soft and hard bottom habitats in the estuarine subtidal model). Section 3.2 has been expanded and enriched to better describe the delineation of ecosystem types as well as the alignment of multiple classifications (CMECS, Cowardin, etc.).

Comment 1.2: Section 4.2.1 - Why does the freshwater tidal zone suitability index increase with an increase in time greater than 0.5 PSU?

Basis: If the theory is that periodic tidal influx of saltwater is beneficial to freshwater marshes through processes of sediment and nutrient deposition, but an excess of saltwater can have a harmful effect on freshwater emergent vegetation, then shouldn’t the index curve be bell-shaped (i.e., peaking at moderate salinity durations and dropping off at high and low durations)?
Significance: Medium.
Recommended Resolution: Provide a justification for the current curve or revise the salinity index per the above logic.
Author Response: Concur The suitability curve was incorrect and has been amended to align with the logic of low levels of salinity as a stimuli and high levels of salinity as a source of harm.

Comment 1.3: Section 4.3.1 - Justify the assumption that mean current velocity and estuarine edge erosion rates are directly correlated.

Basis: Is it possible to directly measure and model edge erosion (possibly from remote sensing such as aerial or satellite imagery)? It seems possible that in some instances (due to certain physical and biological aspects of a particular estuary) increasing velocities could potentially increase sediment accretion and decrease edge erosion (possibly by mobilizing larger quantities and larger grained sediments from outside of the estuary and depositing them into upper estuarine environments). I understand the logic and the simplicity of modeling edge erosion this way, but tidal level and wave action play important roles in this dynamic too.
Significance: Medium.
Recommended Resolution: Not necessarily recommending a change but merely suggesting taking some time to think about this index and whether there is a way to improve it within the limitations of time, budget, modeling constraints, etc.
Author Response: Concur. Direct measurement of erosion would be preferable. However, there were not available methods for long-term forecasting of erosion metrics like shoreline retreat. Changes in velocity were viewed as a compromise between availability and accuracy. Notably, we have changed the focal velocity from an average velocity to the 90th percentile velocity to better reflect the timing of erosion (i.e., during storm conditions).

Comment 1.4: Section 4.3.4 - Is there a well justified reason that the estuarine intertidal land use suitability index relationship is it not linear?.

Basis: Why would 50% urban development receive a score of 1.0, and further why would 50% urban development receive the same score as 0% development? It seems to me like the relationship should be linear with a slope of -1x, with 0% urban development receiving a score of 1.0, and 100% urban development receiving a score of 0.0. Notably, there may be reasons for not wanting to penalize estuaries and marshes due to their proximity to urban development.
Significance: Low.
Recommended Resolution: Provide a justification in the text.
Author Response: Concur. The model initially used a threshold of 50% urban development to avoid penalizing marshes in urban areas. However, the threshold was reduced to 10% to better align with observed declines in ecological condition in the region and general changes in performance in other ecosystems (e.g., stream ecosystems typically degrade beyond ~10% urban development). Additional text was added to justify this change.

Comment 1.5: Section 4.4 - “First, it was assumed that essential fish habitat would be present if oysters, SAV, and clams were present, so this variable was removed”. Revisit this assumption.

Basis: NOAA EFH is a species-specific habitat characterization. So, SAV habitat is EFH for only certain species of fish. For some species, muddy or sandy bottoms are EFH, for some cobble or ledge habitat is EFH. It also depends on the life stage of the species in question (e.g., use of SAV for nursery grounds but not for adult foraging, etc.).
Significance: Medium.
Recommended Resolution: Maybe I’m misunderstand the assumption being made but regardless I would suggest reading, incorporating, and referencing the EFH source documents for the focal taxa that the model is using as surrogates .
Author Response: Concur. This comment was intended in the colloquial use of essential fish habitat rather than the regulatory definition of EFH. The intent was to state that fish habitat quality was generally reflected by the indicator taxa. The text has been clarified accordingly.

Comment 1.6: Section 4.4 - “Here, we use a historical map of oyster reef extent as a proxy for hard bottom with the assumption that hard substrate is relatively immobile and these locations would likely have higher potential for oyster reestablishment”. What about gravel, cobble, and ledge habitat?

Basis: Are these types of geologic hard-bottom habitats present within the study areas? If so, I think they should be included. Cobble and ledge habitat have high ecological and commercial importance and should probably be considered. If a hurricane barrier project alters the amount of hard-bottom, then that would be important to know, especially because a lot of coastal New Jersey and the New York Bight are soft-bottom habitat. For example, ledges provide habitat for kelp, benthic macroalgae, attaching sessile organisms, lobsters, and many fish species. Cobble reefs serve as a nursery for juvenile lobsters and cod. This seems important for an ecological model.
Significance: Medium.
Recommended Resolution: Incorporate geological substrate or explain that the estuarine, subtidal zone is going to be represented with focal taxa, and that hard-bottom is going to be represented by oyster cultch even though other hard-bottom habitat types exist.
Author Response: Concur. Data availability limited our capacity to include these habitat types, and the text was clarified as follows to reflect this assumption. “Additional substrates such as gravel, cobble, or ledge provide important habitat and ecological functions, but these habitats were not considered due to a lack of data availability for the large focal area.”

Comment 1.7: Section 4.4.2.1 - Clarify whether SAV represents vascular plants, marine benthic macroalgae, or both?

Basis: Both seagrasses and benthic macroalgae fall under the category of SAV but require different substrates for ideal habitat. Seagrasses require silt/sand for root expansion and nutrient uptake, while benthic macroalgae requires coarse substrates for attachment (shell reefs, shell rubble, cobbles, boulders, ledge, etc.) . In section 4.4.2.1, the authors write that “substrate is represented as the presence of soft-bottom sediments conducive for SAV growth. Based on Short et al. (2002), optimal conditions for growth were identified as non-cobble substrates with less than 70% fine sediment”. The issue is that the Short et al. (2002) paper is solely dealing with Zostera marina habitat suitability, not benthic macroalgae habitat suitability. However, many studies have found that benthic macroalgae plays an important role in photosynthesis, nursery habitat, habitat structure, etc.
Significance: Medium.
Recommended Resolution: Consult definitions of SAV such as those in encyclopedia.com and from NOAA. The NOAA definition mentions macroalgae but focuses on seagrass. Consider breaking out SAV into two groups (seagrass and benthic macroalgae) or revise the text to be specific to seagrasses and other vascular plants.
Author Response: Concur. Section 4.4.2.1 was amended to clarify the generality of the term SAV and the focus on seagrass in this model.

Comment 1.8: Section 4.5 - Sea turtles shouldn’t be mentioned in the marine, intertidal zone module.

Basis: Sea turtles typically nest in the very upper reaches of the intertidal zone or above the intertidal zone, otherwise their eggs could be smothered in water (embryos unable to respire) or washed away. But more importantly, there aren’t any sea turtles that spawn north of the Carolinas on the Atlantic Coast. Sea turtles are certainly found in the waters off of New York and New Jersey in the Summer and Fall, but not usually in the intertidal zone. Any turtles found on beaches in the intertidal zone are likely resting or stranded. The Dunkin et al. (2016) paper is focused on a study site in southern Florida, so it shouldn’t be used for NYBEM.
Significance: Medium.
Recommended Resolution: Remove sea turtle nesting from habitat suitability.
Author Response: Concur. The reference was initially included as an example structure of a habitat suitability model, and the numerical models did not draw directly from the suitability indices. The citation was removed accordingly.

Comment 1.9: Section 4.5 - The marine, intertidal zone module is solely focused on beach habitat.

Basis: The use of beach slope and vegetation cover make the marine, intertidal zone module a metric of sandy beach quality. What about natural and man-made hard-bottom intertidal areas?
Significance: Medium.
Recommended Resolution: This is probably fine, because my understanding is that most of the intertidal areas of concern within the studies areas are sandy beaches. I would suggest maybe just mentioning this in the text.
Author Response: Concur. The focus on sandy beaches was clarified at the beginning of the section.

Comment 1.10: Section 4.5.4 - Vegetation cover doesn’t seem like an appropriate index for the suitability of marine, intertidal habitat.

Basis: Beach grasses and shrubs only occur in the upper reaches of the intertidal zone, so I’m not sure if it should be used as a metric for the entirety of the marine intertidal zone? In some cases, vegetative cover is preferred by certain species at certain times, such as piping plovers, which often use beach grasses to hide their chicks after they’ve hatched. Many colonial nesting shorebirds (like terns) prefer open sandy habitat but that doesn’t mean all shorebirds do. Many shorebirds nest in beach grasses and shrubs, such as Spotted Sandpipers, Willets, and others.
Significance: Medium.
Recommended Resolution: I would suggest rethinking and updating the marine, intertidal vegetation cover index.
Author Response: Concur. The suitability curve was removed from the analysis to avoid conflicting suitability needs in this ecosystem.

Comment 1.11: Section 4.6.2 - Why does marine subtidal habitat suitability decrease linearly with percent fine substrate?

Basis: The ecological logic of this metric is unclear. Subtidal habitats with fine sediments can be optimal habitat for many important filter-feeding benthic invertebrates and thus provide important foraging habitat for demersal fish species (e.g., winter flounder, cod, etc.). Some undisturbed soft-bottom subtidal habitats can have high benthic invertebrate diversity and density. At the same time, so can hard-bottom habitats or sandy habitats, they just have different communities, not necessarily better ones. More important than grain-size is habitat heterogeneity. Having a mix of different benthic habitats that support different communities and different life stages of benthic and demersal species is more optimal than having just one type of habitat type regardless of its perceived suitability. Additional information can be found in Kritzer et al. 2016.
Significance: Medium.
Recommended Resolution: This index is probably centered around hard clam sediment suitability, but hard clams may not represent the entire marine subtidal ecosystem well. Reconsider the substrate index in the marine, subtidal zone module, but at the very least mention that it is centered on hard clam suitability.
Author Response: Concur. The section was reworded to indicate the focus on hard clam habitat. The model was also modified to reflect a suitability index of 0.1 when fines equal 100%. The broader challenge of incorporating a mosaic of habitats was not feasible within the confines of this patch-based model.

Comment 1.12: Section 4.6 - The Marine, Subtidal Module is essentially focused on only two taxa: hard clams and seagrasses.

Basis: The statement: “NYBEM’s marine subtidal submodel provides a useful framework for estimating general ecosystem condition” is contradicted by the fact that it is only focused on the suitability of hard clams and seagrass. What about other benthic invertebrates? What about coastal fishes?
Significance: High.
Recommended Resolution: The marine, subtidal zone module either needs to be expanded to include a suite of taxa or be heavily caveated in the text.
Author Response: Concur. Section 4.6.5 was heavily revised to reflect the limitations of this simple model.

Comment 1.13: Section 4.7 - The marine, deepwater zone should probably include metrics such as dissolved oxygen, pH, water temperature, turbidity etc.

Basis: From an ecological perspective, the indices used for the marine, deepwater zone are not adequate to predict the suitability of that particular habitat type. I understand that the metrics used in the model are more readily available than some of those proposed here, but it would be beneficial to attempt to incorporate some of these metrics. For example, sea surface temperature (SST) data are available from NOAA, although the direct tie into the model is not clear. If a hurricane barrier caused less influx of cool deepwater water into an estuary, then it could cause SST in the estuary to rise which would have a negative effect on many taxa, including seagrasses. Rising SST is also correlated with algal blooms, carbon release from the seafloor, and many other phenomena.
Significance: High.
Recommended Resolution: If possible, expand the Marine, Deepwater Zone module.
Author Response: Non-concur. The authors agree with the principle of the comment. However, these metrics were not incorporated due to patchy availability of existing data through space and time (i.e., only at monitoring sites) and a general inability to forecast these metrics through time. For instance, SST would provide more refined predictions, but this would require a family of water quality models not currently being developed for this study.

Comment 1.14: The NYBEM would benefit from additional input from estuarine and coastal marine ecologists from academia, non-profit organizations, state, and federal agencies, etc. In particular, addition input on the most appropriate focal / surrogate taxa for each sub-model would be helpful, and sub-models could then be built from those particular taxa.

Basis: Often while reading through the NYBEM documentation, the question of “suitable habitat for what taxa” would arise? The idea of suitable habitat is a species-specific concept (e.g., suitable subtidal habitat for lobster or blue crab is vastly different than subtidal habitat for striped bass or bluefish). Seagrass and hard clams may be appropriate focal taxa, but their use in NYBEM may be overstated. Sub-models such as the marine, subtidal zone and the marine, deepwater zone could be improved by adopting different focal taxa and building indices accordingly. Federally endangered taxa may be an appropriate starting point, given the use for federal projects.
Significance: High.
Recommended Resolution: NYBEM could be constructed around 1-3 focal taxa for each sub-model, which are clearly specified in the text and from which the indices are clearly related. This approach would potentially alleviate a lot of the comments above.
Author Response: Non-concur. NYBEM was constructed in the method recommended. A series of workshops with technical stakeholders were held in 2019 to identify key processes and indicator taxa. The models presented here represent a blended approach, which uses proxy variables from numerous sources and balances those variables with a need to assess outcomes across a large spatial scale.

C.2 Reviewer-2

Comment 2.1: Section 3.2 and 4.1 - “The ecosystem types were adopted from a combination of two existing classifications: (Cowardin et al. 1979; USFWS 1997).” Adopt existing ecosystem classifications.
- Basis: The use and modification of less broadly used systems creates confusion and leads to some metrics being applied inappropriately, particularly with the use of salinity, which is very variable, as a classification factor. The model output in Section 6.3 shows what would generally be classified as estuarine habitat as marine habitat.
- Significance: High. The outputs would be more readily understandable to a wider audience if the model used a standard classification system.
- Recommended Resolution: Change to Cowardin et al. (1979) or CMECS in any future iterations of the model.
- Author Response: Non-concur. See response to Comment 1.1. The classification used here was a blend of the two recommended classifications. Section 3.2 was modified extensively to better describe the rationale.

Comment 2.2: Section 4.0 - “Finally, a numerical “suitability index” was developed for each variable remaining in the sub-model, which were based on existing suitability indices, published thresholds / responses, and professional judgment.” The definition of the suitability index is unclear.

Basis: Throughout the model documentation, the text refers to the suitability index. Suitability implies that the result will answer how suitable the habitat is for a particular thing (suitability = right or appropriate for a particular person, purpose, or situation), but that’s generally not well defined in the text. The question is, “What is the cover type particularly suited for if it gets a 1.0 optimal rating?” It would be more appropriate to refer to the y axis output as a quality index as is done here: “Where Ifresh.tid is an overarching index of ecosystem quality for the freshwater, tidal zone,” but, even in that case, specifically what the index means should be clearly and completely defined. One possibility given the nature of the alternatives with the most potential significant effects – i.e., storm surge barriers – would be to develop and call the index a resilience index. That may support the salinity, relative depth and episodic sediment deposition metrics.
Significance: High. It’s critical for users to understand model outputs. Defining what is meant by suitability or quality would also guide the model team in developing appropriate inputs.
Recommended Resolution: The model outputs should be defined. Define what suitability (or quality) means throughout the model documentation.
Author Response: Concur. Section 4.0 was modified to clarify that suitability is used herein as a generalized notion of ecosystem condition scaled from 0 to 1. The term “suitability” is adopted from the habitat modeling literature due to familiarity of use in the USACE.

Comment 2.3: Section 4.2.1 - “Where Ifresh.tid is an overarching index of ecosystem quality for the freshwater, tidal zone, salinity is a suitability index relative to salinity, veg.cover is a suitability index relative to vegetative cover, and deposition is a suitability index relative to episodic deposition of sediment. All indices are quality metrics scaled from 0 to 1, where 0 is unsuitable and 1 is ideal.” The documentation doesn’t provide sufficient reasoning for metrics or the curve shapes.

Basis: The documentation should explicitly state what the suitability index supports and provide the reasoning for the shape of the lines in the graphs. The justification for the salinity metric is that “Salinity concentration influences many chemical and physical ecological processes within the tidal freshwater ecosystem and supports euryhaline organisms.” The salinity metric graph seems to be upside down; if a marsh is above 0.5 psu 20-100% of the time it gets an SI of 1, but that would mean it’s not a tidal freshwater marsh (TFW) marsh. The SI also seems to be an index of sustainability of the TFW marsh measured by salinity. It’s hard to imagine how the salinity metric will be applied to the project alternatives. Under existing conditions, all TFW marshes would get the highest rating, otherwise, they wouldn’t be TFW marshes. At some point, under the no action alternative and sea level rise, the estuarine zone would migrate upstream so the TFW marsh would become salt marsh and any upstream non-tidal freshwater marsh would become TFW marsh. That’s not necessarily a bad thing, and I don’t see a way for the model to pick up that change. With a floodgate project, the salinity will also creep up with increasingly frequent inundation under normal astronomic tides, but the marsh won’t experience as many or as high of storm tides. With the floodgate, the marsh won’t necessarily need the pulses of sediment from coastal sources (if they get up that far) because the normal tide elevations eventually won’t increase depending on the water elevation when the gates will be closed and the speed and height of SLR. It seems like it would be a pretty complex thing to figure out the percent of the time that the marsh is flooded with water greater than 0.5 psu that would have an adverse effect, especially on the low end of the graph – e.g., the difference between 5% and 15%. TFW marshes are defined based on average annual salinity.
Basis: The documentation doesn’t seem to support the metrics sufficiently or be explicitly linked to project decision making.
Significance: High.
Recommended Resolution: Salinity may not be an appropriate measure of quality. It’s more a measure of what type of marsh it is or sustainability of TFW marsh. I’m not sure this model is the way to evaluate changes in salinity with the project alternatives, so I’d either provide additional reasoning and documentation or drop this metric.
Author Response: Concur. The suitability curve was incorrect and has been amended to align with the logic of low levels of salinity as a stimuli and high levels of salinity as a source of harm. The salinity metric is not intended to capture habitat switching as described, which is accounted for in the habitat zonation algorithms (Sections 3.2 and 4.1). The model is agnostic to the management actions, and instead simply asks, “If this patch is a freshwater tidal zone, how does salinity influence the condition of the patch?” Finally, the authors agree that salinity modeling is challenging, and the NYBEM does not attempt to mechanistically model salinity, but instead adopts salinity forecasts from the Adaptive Hydraulics (AdH) software and simulations.

Comment 2.4: Section 4.2.2 - “For the NYBEM tidal freshwater ecosystem submodel, vegetation cover is quantified as the percentage of emergent aquatic vegetation. Emergent aquatic vegetation is a key habitat resource for species found within this ecosystem like marsh wren, and marsh wren are a key indicator species of ecosystem health because they are drawn to ideal wetland conditions. These important avian taxa are highly territorial and have a minimum habitat area of 50% emergent vegetation coverage (Gutzwiller and Anderson 1987). Marsh wrens rarely breed in marshes with less than 57% emergent vegetation (Gutzwiller and Anderson 1987). As a result, if there is less than this quantity of wetland habitat (emergent vegetation), the suitability index is presumed to be 0.” Modify suitability curve and/or clarify documentation to reflect focus on marsh wren.

Basis: There is insufficient justification for the shape of the graph line. Above this section the documentation says that, “veg.cover is a suitability index relative to vegetative cover.” The documentation would be clearer if it said, “veg.cover is a suitability index that indicates [the capacity of the marsh to support marsh wrens],”or whatever its specific purpose is. Here, the text is implying that veg.cover is an index of habitat quality for marsh wrens. This puts a lot of weight on this single variable for a single species. The graph is only explicitly trying to represent one species and there are many more using this habitat type. 50% and less vegetation cover is certainly not worthless habitat and a marsh with 50% vegetative cover is optimum for some species and general abundance of species. A quick search on hemi-marsh comes up with this from Audubon: “A hemi-marsh is a type of marsh that is roughly equal parts open water and emergent vegetation or plant life. Since hemi-marsh inherently provides a diverse habitat structure, it attracts a variety of birds and other wildlife species.” The graph is not shaped correctly considering other species that might use the habitat type, but I don’t think we can draw a graph that will be useful in this situation. I think the team has to consider that what’s there on a site is good and then project future changes and determine whether they are good or not. The team could consider fully incorporating and applying several HEP models, but I’m not sure that would be worth the effort given the nature of the changes casued by the project alternatives. I would not say that marsh wrens are a key indicator species of ecosystem health because they are drawn to ideal wetland conditions. They are drawn to ideal habitat conditions just like any other species and settle for the habitat that is available to them as long as it meets their minimum requirements.
Significance: High.
Recommended Resolution: The team could consider fully incorporating and applying several HEP models, but I’m not sure that would be worth the effort given the nature of the changes with the project alternatives.
Author Response: Concur. The suitability index was modified to consider a more general notion of vegetation-to-water ratios, and the text was augmented accordingly.

Comment 2.5: Section 4.2.3 – “Episodic sediment deposition requires large magnitude flooding events beyond typical tidal inundation. As such, we develop a metric to assess the relative difference in depth beyond the common tidal datum of MHHW (see equation below). This relative depth metric goes to zero when there is no flooding beyond MHHW, and the metric equals one when flood magnitude is equal to MHHW.” The documentation fails to establish the theoretical basis for this metric and the formula appears to be incorrect.

Basis: The documentation does not establish the theoretical basis for the relative depth index, and there seem to be problems with the formula. The documentation needs to describe what Hmedian and Hmax are. If my check is right, if Hmedian (the elevation of a particular patch?) equals HMHHW, Hrel is 0 and the SI is 1 no matter what the Hmax is. Also, the same amount of flooding (Hmax) over a lower point on the marsh gets a lower SI than a higher point. It’s not clear why that would be. The documentation also doesn’t describe why the sediment deposition SI maxes out at a about 0.05 Hrel. More importantly than the formula, the documentation should provide the rationale for why the relative depth would matter. I could see the actual depth mattering up to a point. Most marshes are pretty flat (certainly high salt marshes are relatively flat) and the hydraulic model results from whatever hydraulic model is being run for the project will indicate the depth of flooding over any patch of marsh. As the documentation says, “increased inundation in tidal freshwater marshes leads to more inorganic sediment deposition, which can assist tidal wetlands keep up with rising sea levels,” but wouldn’t it be simpler to indicate the depth of flooding necessary to maintain sufficient sediment deposition and above which detrimental effects may occur. I believe a hydraulic engineer (or hydraulic model) could estimate these thresholds based on sediment concentration and shear stress and other effects of inundation. The usefulness of this metric should be considered relative to the anticipated changes in tidal hydraulics with the floodgate alternatives under the future without project conditions, project alternatives, and sea level rise scenarios and their effects on marsh sustainability.
Significance: High.
Recommended Resolution: The usefulness of this metric should be considered relative to the anticipated changes in tidal hydraulics with the floodgate alternatives under the future without project conditions, project alternatives, and sea level rise scenarios and their effects on marsh sustainability and revise or delete the metric appropriately.
Author Response: Concur. The definition of the relative depth metric was clarified, and the metric was rederived to align more clearly with suitability. This approach to sediment deposition is linked directly to hydrodynamic change that could result from storm surge barriers.

Comment 2.6: Section 4.2 - Reconsider the overall formula for freshwater tidal habitat suitability.
- Basis: The overall formula Ifresh.tid = (salinity+veg.cover+deposition)/3 seems to be mixing metrics that support different components of TFW marsh quality. Veg.cover seems to be based on habitat value while salinity and deposition appear to be based on sustainability. I’m not sure it makes sense to combine them into one index.
- Significance: High.
- Recommended Resolution: Review and revise the formula as appropriate.
- Author Response: Non-concur. The NYBEM seeks to develop generalized metrics of ecosystem condition that reflect multiple aspects of ecological structure and function. The formula for this system provides one such example.

Comment 2.7: Section 4.3.1 Edge Erosion - “We assume that any increase in average velocity beyond 10% is detrimental, and that increases beyond 30% would fundamentally alter the character of a given marsh.” Some theoretical justification should be provided for this metric and statement.

Basis: Edge erosion will depend on the characteristics of the marsh and the composition of the sediment, and since it’s a percentage metric, existing current speeds. The potential outcomes are that: (a) velocity stays the same under existing conditions = 1.0 SI, (b) velocity increases in the FWOP without a tide gate at some point decreasing the SI, and (c) velocity may increase less with a tide gate giving a lower decrease in SI than FWOP or an increase in the SI. Given the potential outcomes and the fact that this may say that having the barrier will have a better ecological outcome for this metric, it should have sufficient justification.
Significance: High.
Recommended Resolution: Provide theoretical justification for the breakpoints in the curve.
Author Response: Concur. Text was augmented to bolster this description. See also response to Comment 1.3.

Comment 2.8: 4.3.2 Vegetation Cover – This metric also needs more explanation other than the fact that it is adopted from a comparable suitability curve from the Wetland Value Assessment in the Gulf Coast.
- Basis: Need to know the basis for the shape of the line in the graph. Ponds and pannes on salt marshes add wildlife value. I assume that’s not a consideration for this model and that it’s intended to document change as the result of changes in hydraulics with the floodgate alternatives. Any assumptions about coverage of ponds and pannes should be explained and/or reflected in the graph. The documentation should also explain how it considers any deterioration in marsh vegetation due to sea level rise. It would help to indicate where the inputs for these metrics come from – i.e., for this one, the documentation should describe how the predictions about changes in percent cover are being made to be input to this model.
- Significance: Medium.
- Recommended Resolution: Provide additional justification for the shape of the line and where inputs come from.
- Author Response: Concur. Text was augmented to better describe the logic of this variable.

Comment 2.9: 4.3.3 Episodic Sediment Deposition – see comments on section 4.2.3. This section describes the complex interactions of sediment transport on marshes, but settles on a simple metric without and transitional explanation.
- Basis: This section should contain the reasons or basis for why you have the concern described above.
- Significance: High.
- Recommended Resolution: See comments on section 4.2.3.
- Author Response: Concur. Text edited accordingly. See response to Comment 2.5.

Comment 2.10: 4.3.4 Development of Adjacent Upland - The documentation doesn’t provide sufficient justification for this suitability curve.

Basis: The model documentation does not explain how the adjacent development affects the quality of the salt marsh. Is it about “the ability for multiple taxa to use the shoreline as migratory pathways” or is it about allowing for marsh expansion with SLR? The SLAMM Model assesses the capacity of salt marshes to migrate with SLR. “For the NYBEM, habitat suitability is modeled as a function of urban development for adjacent uplands. When the percentage of adjacent urban land uses is greater than 50%, habitat suitability in the estuarine intertidal ecosystem declines. When the development of adjacent upland reaches 100% of the estuarine intertidal ecosystem, the habitat is no longer be considered suitable.” The first sentence needs a citation or additional information to support its use. The documentation should say what it is being considered suitable for. Salt marshes within 100 meters of development may have more value in terms of environmental justice values, and as much value for aquatic and semi-aquatic organisms. This means that any patch of salt marsh or intertidal flats within 100 meters of development will be valued lower, but patches within the same marsh >100 m from the development will be valued higher. Is that practical? Also, it’s not clear how this metric would change with the various scenarios – FWOPC, structures, floodgates, and, given the nature of the areas to which the model will be applied (most likely increasing the size and height of existing structures), maybe it’s not necessary.
Significance: High.
Recommended Resolution: Provide clearer documentation or delete the metric.
Author Response: Concur. The model initially used a threshold of 50% urban development to avoid penalizing marshes in urban areas. However, the threshold was reduced to 10% to better align with observed declines in ecological condition in the region and general changes in performance in other ecosystems (e.g., stream ecosystems typically degrade beyond ~10% urban development). Additional text was added to justify this change. See also response to Comment 1.4.

Comment 2.11: 4.3.5 Shoreline Armoring – “For the NYBEM, the distance to the nearest armored shoreline is used to derive information about the ability for multiple taxa to use the shoreline as migratory pathways.” Additional description of the logic is required.

Basis: There’s a lot of description of shoreline armoring in the documentation, but then the model settles on interruption of migratory pathways as the reason for the graph and formula. Therefore, suitability in this case means the capacity of the shoreline to allow animals to travel between the estuarine habitat (e.g., salt marsh, mudflat, beach) and the adjacent uplands. The reasoning would then be that wildlife that can’t migrate between the estuarine habitat and the upland habitat. Somehow that makes the quality of the marsh half as high as habitats without armoring within the first 100 meters. The justification for the relationship needs to be provided. Since any project would have a structure at all distances (0-100 m) between the structure and the habitat, this should be a bar graph with some value for 0-100 m, rather than a line. It seems like this metric would only provide useful information to assess the effects of shoreline armoring structures. Any NNBF alternative would have to consider the specific effects at the location because the NNBF feature would have to allow for migratory pathways to reduce the effect of armoring – i.e., not just be placed in front an armored shoreline. For a shoreline that already has armoring where the alternative would increase its size, it would not change the rating. For sites without armoring, it would reduce the value of the nearest 100 meters. I don’t think there’s a strong case to decrease the value of a salt marsh because it’s near a structure given the project alternatives.
Significance: High.
Recommended Resolution: Add documentation to support the metric, delete the metric, or consider combining it with the Developed Area metric. I recommend deleting it.
Author Response: Non-concur. The metric is a proxy for many ecological effects of shoreline armoring and human development intensity. Armored shorelines can disconnect migratory paths for animals, inhibit marsh migration, and serve as an indirect metric of other issues like stormwater input. As the reviewer describes, the metric would respond to changes in shoreline based measures like seawalls, but it also serves as a general metric of current condition. The metric is designed as a continuous variable (rather than the categorical bar chart recommended) to avoid unsupported thresholds in model behavior (i.e., a sudden drop in suitability from 101 m to 100 m).

Comment 2.12: 4.4.1 Hard Bottom Habitats - “Here, we use a historical map of oyster reef extent as a proxy for hard bottom with the assumption that hard substrate is relatively immobile and these locations would likely have higher potential for oyster reestablishment.” Revisit the assumption of hard bottom habitat mapping and oysters as a proxy for condition.

Basis: It doesn’t seem like a good idea to use oysters, which are not presently occupying the site to assess the quality of the habitat. I assume there is hard bottom distributed through the estuaries where this will be applied. By applying the salinity requirements for oysters, the model will reduce the value of some hard bottom habitats for a species that’s not even there. It’s also unlikely to be affected by the project alternatives. It’s hard to imagine one of the alternatives seriously affecting hard bottom habitat in a way you could measure and consider the ultimate effects in this model. How can hard bottom be a factor in evaluating hard bottom? Isn’t that a mapping factor? Interspersion is usually considered a good thing for habitat quality. Reducing the quality rating of the habitat because it has other habitats interspersed doesn’t seem to be supported scientifically.
Significance: High.
Recommended Resolution: I suggest dropping this metric unless the model is applied to sites with a lot of oyster habitat.
Author Response: Non-concur. Oyster reefs and hard bottom habitats are an important part of the New York Bight ecosystem, particularly in the New Jersey Back Bays. The reviewer correctly notes the circular logic of delineating the ecosystem based on substrate and using substrate in assessment of ecosystem quality. However, substrate data are limited, and distinguishing these two parallel data sets was not possible for this region. The oyster model was adapted from a prior model and the cultch variable was left in this formulation to avoid confusion with other versions. Notably, the cultch variable was assessed with a binary input for hard bottom, which leads to a suitability of one for all hard bottoms relative to the cultch metric.

Comment 2.13: 4.4.2.1 Submerged Aquatic Vegetation Module - Having the SAV Fine Substrate curve go down to 0.0 SI for 100% fine sediment composition doesn’t seem appropriate. - Basis: See above.
- Significance: High.
- Recommended Resolution: The documentation should provide additional information to support this formula.
- Author Response: Concur. The model was modified to reflect a suitability index of 0.1 when fines equal 100%, and text was edited accordingly.

Comment 2.14: 4.4.2.1 Submerged Aquatic Vegetation Module - “New York State does not have an official standard for Secchi depth, but the state Department of Health requires 1.22 meters (4 feet) of clarity to locate a swimming beach (Lake George Association 2022). This general target aligns with ranges of observations of Secchi depth for the region (DEP 2017).” The documentation should provide justification for applying swimming beach water quality to that needed to maintain eelgrass. - Basis: The whole section on light seems to need more justification. It doesn’t wrap back to a link between the final criteria and eelgrass requirements. The Ecology of Eelgrass Meadows of the Atlantic Coast: A Community Profile has a graph that appears to show optimum PAR and indicates that there are upper limits to suitable levels of PAR for eelgrass that don’t match the NYBEM graph.
- Significance: High.
- Recommended Resolution: Revisit and revise suitability curves for light.
- Author Response: Concur. The logic of the light suitability index was further clarified with a particular emphasis on secchi targets. We specifically highlighted the need to revisit the secchi depth used to estimate Kd. Notably, an upper limit to light exposure was not included in the models in the interest of parsimony and to preserve the emphasis on the more likely condition where lack of limit is a limiting factor.

Comment 2.15: 4.4.2.1 Submerged Aquatic Vegetation Module - Using vessel traffic as a proxy for human disturbance is not sufficiently justified in the documentation. - Basis: There are many instances of federal channels with eelgrass adjacent to and within the channel. Also, I don’t see where the project alternatives will affect vessel traffic – except in the extreme case with sea level rise where the barrier is closed permanently. I recommend deleting this metric.
- Significance: High.
- Recommended Resolution: Revisit model justification for vessel density.
- Author Response: Concur. The justification for vessel traffic was insufficient and has been clarified.

Comment 2.16: 4.4.2.1 Submerged Aquatic Vegetation Module - Reexamine the overarching index equation for SAV. - Basis: If the fine substrate and percent light metrics are correct and sustained, the relationship should be the lowest of substrate.sav or light, rather than adding and dividing by the number of parameters as they would be requirements.
- Significance: High.
- Recommended Resolution: Examine equation and add justification accordingly.
- Author Response: Non-concur. The reviewer is proposing that the equation should reflect limiting factors for SAV growth. The NYBEM is not intended to reflect a particular taxa, and the index equation intends to capture a general condition index for this system.

Comment 2.17: 4.4.2.2 Hard Clam Module – It’s unclear if the NYBEM applies the entire Hard Clam Model. - Basis: The documentation should indicate whether that is the case. (I didn’t see the Thompson reference in the References section. It’s hard to envision how the team will have enough data to determine the salinity at any particular site, but if that’s possible, it’s fine.
- Significance: High.
- Recommended Resolution: Clarify use of Thompson et al.
- Author Response: Concur. The partial use of this model was clarified in the text.

Comment 2.18: 4.4.2.2 Hard Clam Module – This statement doesn’t appear to match the graph or the formula: “Similar to seagrass and oysters, substrate is a critical parameter for hard clam viability. Optimal substrate is shelly soft bottom. As the substrate composition increases in sand and/or mud, the substrate is less suitable for clams.” - Basis: The graph says 100% sand gets the highest SI..
- Significance: High.
- Recommended Resolution: Resolve discrepancy.
- Author Response: Concur. The suitability curve was incorrect, and the model was modified to better reflect response of hard clam to fine sediment composition.

Comment 2.19: 4.4.2.2 Hard Clam Module – The way the soft bottom sub-model is structured, it seems like large areas of the estuaries won’t have a habitat quality rating – maybe that’s ok. - Basis: See above.
- Significance: High.
- Recommended Resolution: Address habitat coverage.
- Author Response: Non-concur. All areas in the estuary receive a habitat suitability index. The estuarine subtidal covers all hard and soft bottom habitats based on hard bottom presence-absence.

Comment 2.20: 4.4.2.2 Hard Clam Module – “As a proxy for seagrass coverage, we use the index of seagrass suitability (Iclam).” You can’t really do this. There are huge areas that look like they could support eelgrass, but don’t.
- Basis: See above.
- Significance: High.
- Recommended Resolution: None.
- Author Response: Non-concur. The connection between the seagrass and hard clam models follows the logic of Thompson et al. (2021). Given a lack of comprehensive seagrass mapping in the region, we use the seagrass habitat suitability as a proxy. The text has been clarified to emphasize this use of a proxy.

Comment 2.21: 4.5.1 Beach Slope – Remove irrelevant references. - Basis: Seabeach amaranth doesn’t grow in the marine intertidal zone. This also seems to include a model for southeast Florida. Need to indicate how alternatives would affect beach slope.
- Significance: High.
- Recommended Resolution: Remove inappropriate citations.
- Author Response: Concur. Seabeach amaranth and sea turtle models were initiatilly included as examples of model structure for this system. Text was edited accordingly.

Comment 2.22: 4.5.2 Exposure Duration – Revise suitability curve to better reflect ecological importance. - Basis: The graph doesn’t appear to be ecologically supported. The areas below the midpoint would be just as valuable as the midpoint, if not more valuable because it is likely to contain more benthic organisms even if it’s not available as long.
- Significance: High.
- Recommended Resolution: Edit suitability curve.
- Author Response: Concur. The range of optimal suitability was expanded to trel from 0.25-0.75.

Comment 2.23: 4.5.3 Upland Land Use – see previous comments on upland land use metric. - Basis: See above.
- Significance: High.
- Recommended Resolution: None.
- Author Response: Concur. See also response to Comments 1.4 and 2.10.

Comment 2.24: 4.5.4 Vegetation Cover – Remove variable from model. - Basis: There shouldn’t be any vegetation (other than algae and maybe extremely rare eelgrass) in the marine intertidal zone. The tern reference applies to beaches, not intertidal.
- Significance: High.
- Recommended Resolution: Remove variable from model.
- Author Response: Concur. See also response Comment 1.10. Variable removed from model.

Comment 2.25: 4.5.5 Shoreline Armoring – see previous comments.
* Basis: See above.
* Significance: High.
* Recommended Resolution: None.
* Author Response: Non-concur. See response to comment 2.11.

Comment 2.26: 4.6 Marine, Subtidal Zone - Suitability curves need additional justification.
* Basis: See above.
* Significance: High.
* Recommended Resolution: None.
* Author Response: Concur. Models were modified according to Comments 1.11 and 1.12, and text was edited for clarity and rationale was bolstered.

Comment 2.27: 4.7 Marine, Deepwater Zone - Suitability curves need additional justification
* Basis: See above.
* Significance: High.
* Recommended Resolution: None.
* Author Response: Concur. Models were modified according to Comment 1.13, and text was edited for clarity and rationale was bolstered.

C.3 Reviewer-3

Comment 3.1: All functions within the nybem R package should have proper documentation, sample data, and provide a basic example of its functionality.

Basis: This is a basic requirement for publishing open source/public R packages to ensure the package and its functions are comprehensible and usable to all users. Additionally, this can help users navigate their own technical challenges by providing a working example with appropriate syntax.
Significance: High.
Recommended Resolution: Create a vignette with working example(s) for each major nybem function, using sample data embedded into the package itself. For these basic examples, users should not be expected to pull data from other sources (R packages, repos, websites, etc.).
Author Response: Concur. All functions are written to include standard format and help documentation. Test data and a testing record are included within the tests folder of the package.

Comment 3.2: SIcalc code and documentation.

Basis: The “approx.” function used in the SIcalc function seems to appropriately interpolate between the SI break points, but this might be initially difficult for users to determine since it is presented quite differently in the documentation vs. the R package/vignette.
Significance: High.
Recommended Resolution: Be explicit in documentation on which set of input variables are defined for each model. For example, nybem_submodels$NYBEM.mar.int documentation shows that “exp dur” with a range of 0-1 is expected, but “trel”, with a range of 0-100 used. This is not the same documentation as land.use which has urban.per as input. Just need to be clear and constant about how to interpret these models and what are actually user inputs.
Author Response: Concur. Documentation was revised to reflect the correct variable naming as indicated.

Comment 3.3: Calling variables in SIcalc.

Basis: The nybem submodels require users to specify the input variables in the right order. It might be more appropriate for the function to look for the names of each variable not the index location. Example below:

#obs_1 <- list(beach.slope, exp.dur, land.use, shoreline, veg.cover) 
#SIcalc(SI = nybem_submodels$NYBEM.mar.int, input_proj = obs_1)

#!=

#obs_1 <- list(beach.slope, exp.dur, land.use, veg.cover, shoreline)
#SIcalc(SI = nybem_submodels$NYBEM.mar.int, input_proj = obs_1)

Significance: High.
Recommended Resolution: This might be a bit picky but I really think it is important to choice between a linear model look up table or interpolating between breakpoints. Just be clear and consistent throughout all documentation.
Author Response: Non-concur. The reviewer makes an excellent point about calling variables directly. However, the nybem code was written to mirror variable structure from an existing package (ecorest). Variable calls were double-checked to ensure consistency in look-up referencing.

C.4 Reviewer-4

Comment 4.1: Given the potential phased approach for these studies and the model, we need to make sure we have a good method for tracking the model and model documentation version(s).

Basis: We need to avoid a situation where potential model users, reviewers, etc. are unclear on what version of the model is being used and whether or not that version is certified.
Significance: Medium – potential inadvertent use of uncertified model.
Recommended Resolution: As discussed during our mid-review webex, recommend clear communication in the model documentation of the model version, certification status, any changes made since certification, etc. Also agree with the recommendation of using the CRAN website for version tracking of the R software package(s).
Author Response: Concur. Following review, the certified version of the model will be uploaded to CRAN, and the associated documentation will be peer-reviewed and published as an ERDC technical report. The model code and documents will be clearly labeled as version 1.0.0, and any subsequent revisions will be identified from new version numbering. Web-based model documents will be labeled and linked accordingly.

Comment 4.2: Section 4.4.2.2 indicates that optimal clam substrate is shelly soft bottom and as the substrate composition increases in sand and/or mud, the substrate is less suitable for clams. This is contradicted by the HSI curve which indicates increasing suitability as sand composition increases.

Basis: Contradiction in description of metric vs. HSI curve and formulas.
Significance: Low.
Recommended Resolution: Recommend either modifying text to comport with HSI curve and formulas or vice versa.
Author Response: Concur. Text and figures were aligned.