豪情万千的山羊 · Member 1 power ...· 1 月前 · |
叛逆的沙发 · 【企业级持续集成实战系列】主流持续集成完整实 ...· 11 月前 · |
飘逸的鞭炮 · 蓝牙avrcp是啥 - OSCHINA - ...· 1 年前 · |
慷慨大方的麻辣香锅 · Jmeter 压力测试:性能测试之路 - 知乎· 1 年前 · |
Error Importing repository - No such file or directory @ rb_sysopen - (filename)
Diagnosing prolonged or failed imports
To bring existing projects to GitLab, or copy GitLab groups and projects to a different location, you can:
The best way to copy GitLab groups and projects between GitLab instances, or in the same GitLab instance, is by using direct transfer .
Another option is to move GitLab groups using group transfer .
You can also copy GitLab projects by using a GitLab file export, which is a supported import source.
The import sources that are available to you by default depend on which GitLab you use:
GitLab can import projects from these supported import sources. Import source Description Bitbucket Cloud Using Bitbucket.org as an OmniAuth provider , import Bitbucket repositories. Bitbucket Server Import repositories from Bitbucket Server (also known as Stash). FogBugz Import FogBugz projects. Gitea Import Gitea projects. GitHub Import from either GitHub.com or GitHub Enterprise. GitLab export Migrate projects one by one by using a GitLab export file. Manifest file Upload a manifest file. Repository by URL Provide a Git repository URL to create a new project from.
Only import projects from sources you trust. If you import a project from an untrusted source,
an attacker could steal your sensitive data. For example, an imported project
with a malicious
.gitlab-ci.yml
file could allow an attacker to exfiltrate group CI/CD variables.
GitLab self-managed administrators can reduce their attack surface by disabling import sources they don’t need:
You can also read information on importing from these other import sources:
GitLab can not automatically migrate Subversion repositories to Git. Converting Subversion repositories to Git can be
difficult, but several tools exist including:
git svn
, for very small and basic repositories.
reposurgeon
, for larger and more complex repositories.
importer_user_mapping
and
bulk_import_importer_user_mapping
. Disabled by default.
Introduced to Gitea project import
in GitLab 17.6
with flags
named
importer_user_mapping
and
gitea_user_mapping
. Disabled by default.
This method of user contributions and membership mapping is available for direct transfer migrations on:
For information on the other method available for GitLab self-managed without enabled feature flags, see User contributions and membership mapping .
With user contribution and membership mapping, you can assign imported contributions and memberships to users on the destination instance after import has completed. Unlike the previous method of user contribution and membership mapping, no preparation is needed before the import.
The process doesn’t rely on email addresses, so you can map contributions for users who have different emails on source and destination instances.
Each user on the destination instance that is assigned a mapping can: Explicitly accept the assignment before any imported contributions are attributed to them.
Instead of immediately assigning contributions and memberships to users on the destination instance, a placeholder user is created for any user whose contributions or memberships were imported.
Both contributions and memberships are first assigned to these placeholder users and can be reassigned after import to existing users on the destination instance.
Until they are reassigned, contributions display as associated with the placeholder. Placeholder memberships do not display in member lists.
A placeholder user is created for each user on the source instance, except in the following scenarios:
Placeholder users are different to regular users and cannot:
To maintain a connection with a user on a source instance, placeholder users have:
source_user_id
) used by the import process to determine if a new placeholder user is required.
source_hostname
).
source_name
) to help with reassignment of contributions.
source_username
) to facilitate group owners during the reassignment of the contribution.
import_type
) to distinguish which importer created the placeholder.
To preserve historical context, the placeholder user name and username are derived from the source user name and username:
Placeholder <source user name>
.
%{source_username}_placeholder_user_%{incremental_number}
.
Prerequisites:
Placeholder users are created on the destination instance while a group or project is imported. To view placeholder users created during imports to a top-level group and its subgroups:
Placeholder users are created per import source and per top-level group:
If importing to GitLab.com, placeholder users are limited per top-level group on the destination instance. The limits differ depending on your plan and seat count. Placeholder users do not count towards license limits. GitLab.com plan Number of seats Placeholder user limit on top-level group Free and any trial Any amount Premium Premium 101-500 Premium 501 - 1000 Premium Ultimate and open source Ultimate and open source 101-500 Ultimate and open source 501 - 1000 Ultimate and open source
Customers on legacy Bronze, Silver, or Gold plans have the corresponding Free, Premium, or Ultimate limits. For Premium customers trying out Ultimate (Ultimate trial paid customer plan), Premium limits apply.
If these limits are not sufficient for your import, contact GitLab Support .
The above limits are for GitLab.com. Self-managed GitLab has no placeholder limits by default. A self-managed instance administrator can set a placeholder limit for their installation.
Reassignment of contributions and memberships from placeholder users to existing active (non-bot) users occurs on the destination instance. On the destination instance, you can:
All the contributions initially assigned to a single placeholder user can only be reassigned to a single active regular user on the destination instance. The contributions assigned to a single placeholder user cannot be split among multiple active regular users.
Bot user contributions and memberships on the source instance cannot be reassigned to bot users on the destination instance. You might choose to keep source bot user contributions assigned to a placeholder user .
Users that receive a reassignment request can: Accept the request . All contributions and membership previously attributed to the placeholder user are re-attributed to the accepting user. This process can take a few minutes, depending on the number of contributions. Reject the request or report it as spam. This option is available in the reassignment request email.
In subsequent imports, contributions and memberships that belong to the same source user are automatically mapped to the user who previously accepted reassignments for that source user.
The reassignment process must be fully completed before you: Move an imported group in the same GitLab instance . Move an imported project to a different group .
If the process isn’t complete, contributions still assigned to placeholder users cannot be reassigned to real users and they stay associated with placeholder users.
Once this contribution and membership reassignment is complete, it cannot be undone so check all everything before starting.
Reassigning contributions and membership to an incorrect user poses a security threat, because the user becomes a member of your group. They can, therefore, view information they should not be able to see.
Reassigning contributions to users with administrator access is disabled by default, but you can enable it.
Because of the GitLab permissions model, when a group or project is imported into an existing parent group, members of the parent group are granted inherited membership of the imported group or project.
Selecting a user for contribution and membership reassignment who already has an existing inherited membership of the imported group or project can affect how memberships are reassigned to them.
GitLab does not allow a membership in a child project or group to have a lower role than an inherited membership. If an imported membership for an assigned user has a lower role than their existing inherited membership, the imported membership is not reassigned to the user.
This results in their membership for the imported group or project being higher than it was on the source.
Prerequisites:
To request a user accept reassignment of contributions and memberships:
Contributions of only one placeholder user can be reassigned to an active non-bot user on destination instance.
Before a user accepts the reassignment, you can cancel the request .
You might not want to reassign contributions and memberships to users on the destination instance. For example, you might have former employees that contributed on the source instance, but they do not exist as users on the destination instance.
In these cases, you can keep the contributions assigned to placeholder users. Placeholder users do not keep membership information because they cannot be members of projects or groups .
Because names and usernames of placeholder users resemble names and usernames of source users, you keep a lot of historical context.
Remember that if you keep remaining placeholder users as placeholders, you cannot reassign their contributions to actual users later. Ensure all required reassignments are completed before keeping the remaining placeholder users as placeholders.
You can keep contributions assigned to placeholder users either one at a time or in bulk.
To keep placeholder users one at a time:
To keep placeholder users in bulk:
Before a user accepts a reassignment request, you can cancel the request:
If a user is not acting on a reassignment request, you can prompt them again by sending another email:
You can review statuses of all placeholder users for which the reassignment process haven’t been completed yet:
You can filter by reassignment status:
In the
Awaiting reassignment
tab possible statuses are:
Not started
- Reassignment has not started.
Pending approval
- Reassignment is waiting on user approval.
Reassigning
- Reassignment is in progress.
Rejected
- Reassignment was rejected by user.
Failed
- Reassignment failed.
In the
Reassigned
tab possible statuses are:
Success
- Reassignment succeeded.
Kept as placeholder
- Placeholder user was made permanent.
By default, the table is sorted alphabetically by placeholder user name. You can also sort the table by reassignment status:
You might receive an email informing you that an import process took place and asking you to confirm reassignment of contributions to yourself.
If you were informed about this import process, you must still review reassignment details very carefully. Details listed in the email are: Imported from - The platform the imported content originates from. For example, another instance of GitLab, GitHub, or Bitbucket. Original user - The name and username of the user on the source platform. This could be your name and user name on that platform. Imported to - The name of the new platform, which can only be a GitLab instance. Reassigned to - Your full name and username on the GitLab instance. Reassigned by - The full name and username of your colleague or manager that performed the import.
If you receive an email asking you to confirm reassignment of contributions to yourself and you don’t recognize or you notice mistakes in this information:
You must review the reassignment details of any reassignment request very carefully. If you were not already informed about this process by a trusted colleague or your manager, take extra care.
Rather than accept any reassignments that you have any doubts about:
Accept reassignments only from the users that you know and trust. Reassignment of contributions is permanent and cannot be undone. Accepting the reassignment might cause contributions to be incorrectly attributed to you.
The contribution reassignment process starts only after you accept the reassignment request by selecting Approve reassignment in GitLab. The process doesn’t start by selecting links in the email.
You can view all project imports created by you. This list includes the following:
To view project import history:
The history also includes projects created from built-in or custom templates. GitLab uses import repository by URL to create a new project from a template.
When importing a project that contains LFS objects, if the project has an
.lfsconfig
file with a URL host (
lfs.url
) different from the repository URL host, LFS files are not downloaded.
If you prefer, you can engage GitLab Professional Services to migrate groups and projects to GitLab instead of doing it yourself. For more information, see the Professional Services Full Catalog .
Importers rely heavily on Sidekiq jobs to handle the import and export of groups and projects. Some of these jobs might consume significant resources (CPU and memory) and take a long time to complete, which might affect the execution of other jobs. To resolve this issue, you should route importer jobs to a dedicated Sidekiq queue and assign a dedicated Sidekiq process to handle that queue.
For example, you can use the following configuration:
sidekiq['concurrency'] = 20
sidekiq['routing_rules'] = [
# Route import and export jobs to the importer queue
['feature_category=importers', 'importers'],
# Route all other jobs to the default queue by using wildcard matching
['*', 'default']
sidekiq['queue_groups'] = [
# Run a dedicated process for the importer queue
'importers',
# Run a separate process for the default and mailer queues
'default,mailers'
In this setup:
- A dedicated Sidekiq process handles import and export jobs through the importer queue.
- Another Sidekiq process handles all other jobs (the default and mailer queues).
- Both Sidekiq processes are configured to run with 20 concurrent threads by default.
For memory-constrained environments, you might want to reduce this number.
If your instance has enough resources to support more concurrent jobs,
you can configure additional Sidekiq processes to speed up migrations.
For example:
sidekiq['queue_groups'] = [
# Run three processes for importer jobs
'importers',
'importers',
'importers',
# Run a separate process for the default and mailer queues
'default,mailers'
With this setup, multiple Sidekiq processes handle import and export jobs concurrently,
which speeds up migration as long as the instance has sufficient resources.
For the maximum number of Sidekiq processes, keep the following in mind:
- The number of processes should not exceed the number of available CPU cores.
- Each process can use up to 2 GB of memory, so ensure the instance
has enough memory for any additional processes.
- Each process adds one database connection per thread
as defined in
sidekiq['concurrency']
.
For more information, see running multiple Sidekiq processes
and processing specific job classes.
Troubleshooting
Imported repository is missing branches
If an imported repository does not contain all branches of the source repository:
- Set the environment variable
IMPORT_DEBUG=true
.
- Retry the import with a different group, subgroup, or project name.
- If some branches are still missing, inspect
importer.log
(for example, with jq
).
Exception: Error Importing repository - No such file or directory @ rb_sysopen - (filename)
The error occurs if you attempt to import a tar.gz
file download of a repository’s source code.
Imports require a GitLab export file, not just a repository download file.
Diagnosing prolonged or failed imports
If you’re experiencing prolonged delays or failures with file-based imports, especially those using S3, the following may help identify the root cause of the problem:
- Check import steps
- Review logs
- Identify common issues
Check import status
Check the import status:
- Use the GitLab API to check the import status of the affected project.
- Review the response for any error messages or status information, especially the
status
and import_error
values.
- Make note of the
correlation_id
in the response, as it’s crucial for further troubleshooting.
Review logs
Search logs for relevant information:
For self-managed instances:
- Check the Sidekiq logs and
exceptions_json
logs.
- Search for entries related to
RepositoryImportWorker
and the correlation ID from Check import status.
- Look for fields such as
job_status
, interrupted_count
, and exception
.
For GitLab.com (GitLab team members only):
Use Kibana to search the Sidekiq logs with queries like:
Target: pubsub-sidekiq-inf-gprd*
json.class: "RepositoryImportWorker" AND json.correlation_id.keyword: "<CORRELATION_ID>"
json.class: "RepositoryImportWorker" AND json.meta.project: "<project.full_path>"
Look for the same fields as mentioned for self-managed instances.
Identify common issues
Check the information gathered in Review logs against the following common issues:
Interrupted jobs: If you see a high interrupted_count
or job_status
indicating failure, the import job may have been interrupted multiple times and placed in a dead queue.
S3 connectivity: For imports using S3, check for any S3-related error messages in the logs.
Large repository: If the repository is very large, the import might time out. Consider using Direct transfer in this case.
to fix an error or add an improvement in a merge request.
Create an issue
to suggest an improvement to this page.
Product
Create an issue
if there's something you don't like about this feature.
Propose functionality
by submitting a feature request.
Join First Look
to help shape new features.
Feature availability and product trials
View pricing
to see all GitLab tiers and features, or to upgrade.
Try GitLab for free
with access to all features for 30 days.
search the docs.
If you want help with something specific and could use community support,
post on the GitLab forum.
For problems setting up or using this feature (depending on your GitLab
subscription).
Request support
慷慨大方的麻辣香锅 · Jmeter 压力测试:性能测试之路 - 知乎 1 年前 |