The purpose of this guide is to orient scientists and other data users to California’s commercial lobster fishery logbook data. These data are confidential and are only provided through authorized data sharing agreements. This guide seeks to help scientists and other data users anticipate how these data are formatted and how they may have to “clean” (a.k.a., process) the data before they can be used in analysis. I also seek to highlight special tricks and caveats gleaned from my experience using the data. Ultimately, I hope this guide will help clarify what data are available and how these data can be processed to maximize their utility.
Overview
The California Department of Fish and Wildlife (CDFW) has required commercial lobster fishermen to record information on fishing trips in commercial logbooks since 1973. In the mid-1990s, CDFW transitioned from requiring daily logs to requiring logs that could record up to three days of fishing. These logbooks record the date and location of fishing, the vessel, permit holder, and crew conducting the fishing, the number of legal-sized lobsters retained and the number of sublegal-sized lobsters released, the number of traps pulled and currently deployed, the number of nights the pulled traps were set. Location information includes the block id and since 2017, has included GPS coordinates. Before 2017, a location name was provided. The logbooks alaor reporting the id of the associated landings receipt. Each logbook form can record two days of fishing with up to five sets of trap pulls per day.
An example logbook is available here. Additional information is available here.
This guide provides an overview of California’s commercial lobster logbook data based on a data request processed in August 2022 for all lobster logbooks from 2000-2020 (only data after 2005 was shared). In this guide, I review the attributes of the lobster logbook data, the steps required to clean the raw data, and some visualizations of non-confidential summaries of the lobster data.
Data attributes
The CPFV logbook data includes the following columns:
| Column | Description |
| LogDateString | Date |
| SerialNumber | Logbook id |
| VesselID | Vessel id |
| VesselName | Vessel name |
| FisherLicenseID | Fisher license id |
| FisherLastName | Fisher last name |
| FisherFirstInitial | Fisher first name |
| Block | Block id, where most traps were placed |
| Depth | Depth (ft), where most traps were placed |
| LegalsRetained | Number of legal-sized lobsters retained |
| ShortsReleased | Number of sub-legal-sized lobsters released |
| TrapsPulled | Number of traps pulled |
| MultiDayReceivered | Was the trip multiple days or were lobsters received for future sale (Y/N)? |
| NumTrapsDeployed | Number of traps currently deployed in the water |
| LandingReceipt | Landing receipt(s) associated with day’s fishing |
| CrewMemberID | Crew member ids |
| TrapLocation | Location – either a place name (pre-2017) or lat/long (post-2017) |
| LatitudeDec | Latitude (°N) |
| LongitudeDec | Longitude (°W) |
| NightsInWater | Nights in water |
Attribute information and issues
Logbook id
A single logbook can include multiple different fishing activities, where fishing activities are defined by different dates or locations.
Vessel id and name
The vessel id and name fields are generally well-formatted.
However, I had to correct the spelling of one vessel name and add a vessel name to two vessel ids to harmonize vessel id-name combinations in the data. Even after making these corrections, a few vessel ids have two names (e.g., vessels 02856, 06051, etc).
Fisher id and name
The fisher id and names are generally well-formatted.
However, the following fisher ids are not valid: “LE END”, “L”, “LS. C”
I also had to add a first/last name to a single fisher id missing this information in some rows (but with this information provided in other rows).
Location information
The location column is not well formatted.
It provides one of two types of location information: (1) a place name, which is not harmonized throughout; or (2) GPS coordinates that approximate the following format: “32° 15.40′ lat. 117° 39.80′ long.” Note that the minutes are in decimal minutes. In many of the latitude and longitude columns, CDFW incorrectly treated the decimal as “seconds”. This is incorrect. See more in the “Coordinates” section below.
Figure 1. The type of information used to report the location of fishing in lobster logbooks from 2005 to 2020. Lobster fishermen were required to start reporting location in terms of GPS coordinates in 2017. Coordinates are reported using decimal minutes to the hundredths place.
Block id
The block id column is not well-formatted and includes many mistyped ids. It also includes many invalid block ids and valid block ids from which lobsters are not known to be caught.
- Mistyped block ids (note O’s instead of 0s in later ids): 719+, 756’, ‘822, .842, 075+, 68/0, 68/8, 68*9, A0708, 86+1, RR22, O306, O326, O345
- Invalid block ids: 0, 2, 3, 10, 18, 19, 20, 21, 25, 40, 49, 50, 52, 54, 56, 60, 61, 62, 63, 64, 65, 66, 68, 70, 71, 72, 73, 75, 79, 80, 81, 83, 85, 86, 87, 100, 152, 153, 156, 157, 160, 180, 181, 190, 306, 307, 326, 342, 345, 351, 352, 353, 381, 383, 385, 387, 389, 500, 575, 576, 588, 600, 698, 699, 700, 778, 780, 785, 786, 787, 788, 789, 800, 901, 904, 919, 921, 922, 953, 954, 955, 957, 960, 980, 982, 983, 984, 985, 988, 1019, 1028, 3718, 3798, 3860, 3900, 5814, 6001, 6353, 6354, 6355, 6389, 6520, 6532, 6533, 6543, 6654, 6665, 6866, 6885, 6889, 6893, 6894, 6953, 7009, 7102, 7110, 7203, 7230, 7387, 7455, 7520, 7536, 7564, 7569, 7574, 7757, 8136, 8432, 8442, 8460, 8520, 8600, 8605, 8630, 8650, 8660, 8693, 8707, 8760, 8777, 8821, 8822, 8850, 8860, 8922, 8942, 8960
A few block ids listed the longitude of the trap set rather than the block id. I moved the longitude into the location column (longitude was missing from the location column in these cases).
Figure 2. The location of fishing effort as measured as the number of logbooks reporting effort to a block from 2005 to 2020. Commercial lobster fishing is not traditionally thought to occur north of Point Conception or in waters deeper than 300 ft. Note the log-scale in the fill colors.
Coordinates
Since 2017, fishermen have been asked to record the GPS coordinates of trap sets using “decimal minutes to the hundredths place (e.g., 34° 05.15N, 120° 04.85W)”.
This information is reported in the “TrapLocation” column and is then broken out into decimal degrees in the “LatitudeDec” and “LongitudeDec” columns by CDFW.
Unfortunately, CDFW performed these conversions incorrectly. They converted the coordinates to decimal degrees but oddly assumed that the decimal minutes implied seconds. They did this despite the fact that fishermen were explicitly instructed to provide the coordinates in decimal minutes and that the decimal often exceeded 60, which it couldn’t if it reflected seconds.
For example, CDFW converted (32° 00.91′ lat. 118° 36.65′ long.) from logbook “XXXXXXXX” to (32.02528, 118.6181) when they have converted it to (32.015167, 118.61083).
Wrong method: 118°36.65’W = 118 + 36 / 60 + 65 / 3600 = 118.6181
Correct method: 118°36.65’W = 118 + 36.65 / 60 = 118.61083
I harmonized the formatting of the location column and correctly converted the coordinates to decimal degrees.
Despite resolving these issues, many of the trap set locations occur on land (see below).
Figure 3. The location of trap sets reported in lobster logbooks.
Depth (ft)
Fishermen report the depth where the majority of fishing took place. These values are largely reasonable, except for a few large unreasonable values. See the figure below.
Figure 4. The distribution of depths reported in the lobster logbooks. Lobster fishing is generally thought to occur in water less than 300 ft deep. Note the log-scale on the x-axis.
Fishing effort (number of traps and soak time)
Fishing effort is recorded in terms of the number of traps pulled and the number of night in which the traps were left fishing. This allows for the calculation of effort in terms of “trap-nights”.
The number of traps left in the water is also indicated in the logbook. There are some large values in these columns that data users should consider carefully.
Figure 5. The distribution of fishing effort reported in the lobster logbooks in terms of (A) number of nights, (B) number of traps pulled; and (C) number of traps currently deployed. Note the log-scale on the y-axis in (A).
Catch statistics
Fishermen report the number of legal-sized lobsters retained and the number of sublegal-sized lobsters (“shorts”) released. There are occasionally very large values that users should carefully consider when using these data. See the visualizations below for lobster catch per trip and over time.
Figure 6. The distribution of the number of lobsters (A) retained and (B) released in the lobster logbooks. Note the log-scale on the y-axis in (A).
Figure 7. Millions of lobsters retained and released from 2005-2020 based on the lobster logbook data.