Aggiornata la documentazione per gli agents

This commit is contained in:
Alessio Dal Santo
2025-09-12 17:51:50 +02:00
parent 78676d4485
commit b76a6760fb
2 changed files with 921 additions and 591 deletions
+921
View File
@@ -0,0 +1,921 @@
# Data-Coupler: Sistema di Integrazione Dati Multi-Platform
## 📋 Project Overview
**Data-Coupler** è una soluzione enterprise di integrazione dati sviluppata in .NET 9.0 con architettura Blazor Server. Il sistema facilita il trasferimento e la sincronizzazione di dati tra diverse tipologie di sorgenti: database relazionali, REST API, file Excel/CSV.
### 🎯 Obiettivi del Progetto
- **Unificazione Data Sources**: Connessione trasparente a database eterogenei e API REST
- **Trasferimenti Sicuri**: Gestione crittografata delle credenziali cross-platform
- **Automazione**: Sistema di scheduling per operazioni periodiche
- **Mappatura Intelligente**: Sistema avanzato di mapping campi tra sorgenti diverse
- **Gestione Associazioni**: Tracking delle chiavi per evitare duplicati e gestire relazioni
### 🏗️ Architettura del Sistema
#### Struttura Modulare
```
Data-Coupler/
├── Data_Coupler/ # 🎯 Applicazione principale Blazor Server
├── DataConnection/ # 🔌 Libreria core per connessioni dati
├── CredentialManager/ # 🔐 Gestione sicura credenziali
├── Components/ # 🧩 Componenti UI riutilizzabili
└── DatabaseUpdater/ # 🔧 Utility per manutenzione database
```
#### Stack Tecnologico
- **.NET 9.0**: Framework principale
- **Blazor Server**: Framework UI reattivo
- **Entity Framework Core**: ORM per accesso dati
- **SQLite**: Database embedded per configurazioni
- **Bootstrap 5**: Framework CSS responsive
#### Database Supportati
- SQL Server, MySQL, PostgreSQL, Oracle, SQLite, IBM DB2, SAP HANA
#### REST API Supportate
- Generic REST APIs, Salesforce, SAP Business One Service Layer
## 🔨 Build and Test Commands
### Prerequisiti
- .NET 9.0 SDK
- Git per clone del repository
- Editor: Visual Studio 2022, VS Code, o Rider
### Setup Ambiente di Sviluppo
```bash
# Clone del repository
git clone https://github.com/AlessioDalsi/Data-Coupler.git
cd Data-Coupler
# Restore dei pacchetti NuGet
dotnet restore Data_Coupler.sln
```
### Build Commands
#### Build Development
```bash
# Build completo della soluzione
dotnet build Data_Coupler.sln
# Build singolo progetto
dotnet build Data_Coupler/Data_Coupler.csproj
dotnet build DataConnection/DataConnection.csproj
dotnet build CredentialManager/CredentialManager.csproj
```
#### Build Production
```bash
# Build release
dotnet build Data_Coupler.sln --configuration Release
# Publish per deployment
dotnet publish Data_Coupler/Data_Coupler.csproj \
--configuration Release \
--output ./publish \
--self-contained true \
--runtime win-x64
```
#### Build per Diverse Piattaforme
```bash
# Windows x64
dotnet publish --runtime win-x64 --configuration Release
# Linux x64
dotnet publish --runtime linux-x64 --configuration Release
# macOS x64
dotnet publish --runtime osx-x64 --configuration Release
# macOS ARM64 (Apple Silicon)
dotnet publish --runtime osx-arm64 --configuration Release
```
### Run Commands
#### Sviluppo Locale
```bash
# Avvio applicazione principale
dotnet run --project Data_Coupler/Data_Coupler.csproj
# Avvio con hot reload
dotnet watch run --project Data_Coupler/Data_Coupler.csproj
# Avvio su porta specifica
dotnet run --project Data_Coupler/Data_Coupler.csproj --urls "http://localhost:8080"
```
#### Utility e Manutenzione
```bash
# Aggiornamento database
dotnet run --project DatabaseUpdater/DatabaseUpdater.csproj
# Creazione migrazione Entity Framework
cd CredentialManager
dotnet ef migrations add [MigrationName]
dotnet ef database update
```
### Test Commands
#### Esecuzione Test Unitari
```bash
# Esecuzione tutti i test
dotnet test Data_Coupler.sln
# Test con verbosità dettagliata
dotnet test Data_Coupler.sln --verbosity detailed
# Test con coverage
dotnet test Data_Coupler.sln --collect:"XPlat Code Coverage"
# Test specifici per progetto
dotnet test DataConnection.Tests/DataConnection.Tests.csproj
dotnet test CredentialManager.Tests/CredentialManager.Tests.csproj
```
#### Test di Integrazione
```bash
# Test di integrazione database
dotnet test --filter "Category=Integration"
# Test connessioni REST
dotnet test --filter "Category=RestIntegration"
# Test end-to-end
dotnet test --filter "Category=E2E"
```
#### Performance Testing
```bash
# Benchmark performance
dotnet run --project PerformanceTests/PerformanceTests.csproj --configuration Release
# Memory profiling
dotnet run --project Data_Coupler/Data_Coupler.csproj --configuration Release
# Utilizzare strumenti come dotMemory o PerfView
```
## 📝 Code Style Guidelines
### Convenzioni di Naming
#### C# Naming Conventions
```csharp
// Classes e Interfaces: PascalCase
public class DataConnectionFactory { }
public interface IDatabaseManager { }
// Methods: PascalCase
public async Task<List<Data>> GetDataAsync() { }
// Properties: PascalCase
public string ConnectionString { get; set; }
// Private fields: camelCase con underscore
private readonly ILogger _logger;
// Local variables e parameters: camelCase
public void ProcessData(string connectionName, int batchSize) { }
// Constants: PascalCase
public const string DefaultTimeout = "30";
```
#### File e Directory Naming
```
// Cartelle: PascalCase
Services/
Models/
Extensions/
// File C#: PascalCase con classe principale
DatabaseConnectionService.cs
CredentialEntity.cs
// File Razor: PascalCase
DataCoupler.razor
ProfileManagement.razor
```
### Organizzazione Codice
#### Structure dei File
```csharp
using System; // System namespaces
using System.Collections.Generic;
using Microsoft.Extensions; // Microsoft namespaces
using DataConnection.Interfaces; // Project namespaces
using ThirdParty.Library; // Third-party namespaces
namespace Data_Coupler.Services
{
/// <summary>
/// Descrizione chiara della classe
/// </summary>
public class ExampleService : IExampleService
{
#region Private Fields
private readonly ILogger<ExampleService> _logger;
#endregion
#region Constructor
public ExampleService(ILogger<ExampleService> logger)
{
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
}
#endregion
#region Public Methods
public async Task<Result> ProcessAsync(Request request)
{
// Implementation
}
#endregion
#region Private Methods
private void ValidateRequest(Request request)
{
// Validation logic
}
#endregion
}
}
```
#### Async/Await Pattern
```csharp
// ✅ Corretto
public async Task<List<Data>> GetDataAsync(CancellationToken cancellationToken = default)
{
var data = await _repository.GetDataAsync(cancellationToken);
return data.ToList();
}
// ❌ Evitare
public async Task<List<Data>> GetData()
{
var data = await _repository.GetDataAsync().Result; // Blocking call
return data.ToList();
}
```
#### Error Handling
```csharp
// ✅ Gestione errori strutturata
public async Task<Result<Data>> ProcessDataAsync(string input)
{
try
{
if (string.IsNullOrWhiteSpace(input))
return Result<Data>.Failure("Input cannot be empty");
var data = await _service.ProcessAsync(input);
return Result<Data>.Success(data);
}
catch (ValidationException ex)
{
_logger.LogWarning(ex, "Validation failed for input: {Input}", input);
return Result<Data>.Failure(ex.Message);
}
catch (Exception ex)
{
_logger.LogError(ex, "Unexpected error processing input: {Input}", input);
return Result<Data>.Failure("An unexpected error occurred");
}
}
```
### Dependency Injection Guidelines
#### Service Registration
```csharp
// Program.cs - Registrazione servizi
// Singleton per servizi stateless
builder.Services.AddSingleton<IEncryptionService, EncryptionService>();
// Scoped per servizi con stato per request
builder.Services.AddScoped<IDataConnectionFactory, DataConnectionFactory>();
// Transient per servizi leggeri
builder.Services.AddTransient<IValidator<Model>, ModelValidator>();
```
#### Constructor Injection
```csharp
public class ProfileService
{
private readonly IDbContext _context;
private readonly ILogger<ProfileService> _logger;
private readonly IOptions<AppSettings> _options;
public ProfileService(
IDbContext context,
ILogger<ProfileService> logger,
IOptions<AppSettings> options)
{
_context = context ?? throw new ArgumentNullException(nameof(context));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
_options = options ?? throw new ArgumentNullException(nameof(options));
}
}
```
### Commenti e Documentazione
#### XML Documentation
```csharp
/// <summary>
/// Trasferisce dati da una sorgente a una destinazione utilizzando il mapping specificato
/// </summary>
/// <param name="sourceConnection">Connessione alla sorgente dati</param>
/// <param name="destinationConnection">Connessione alla destinazione</param>
/// <param name="mapping">Configurazione mapping campi</param>
/// <param name="cancellationToken">Token per cancellazione operazione</param>
/// <returns>Risultato del trasferimento con statistiche</returns>
/// <exception cref="ArgumentNullException">Lanciata quando un parametro richiesto è null</exception>
/// <exception cref="InvalidOperationException">Lanciata quando le connessioni non sono valide</exception>
public async Task<TransferResult> TransferDataAsync(
IDataConnection sourceConnection,
IDataConnection destinationConnection,
FieldMapping mapping,
CancellationToken cancellationToken = default)
{
// Implementation
}
```
## 🧪 Testing Instructions
### Struttura Test
#### Test Project Organization
```
Tests/
├── Unit/ # Test unitari
│ ├── Data_Coupler.Tests/
│ ├── DataConnection.Tests/
│ └── CredentialManager.Tests/
├── Integration/ # Test di integrazione
│ ├── Database.Integration.Tests/
│ └── API.Integration.Tests/
└── E2E/ # Test end-to-end
└── UI.Tests/
```
### Test Unitari
#### Setup Test Class
```csharp
[TestClass]
public class CredentialServiceTests
{
private Mock<IDbContext> _mockContext;
private Mock<IEncryptionService> _mockEncryption;
private Mock<ILogger<CredentialService>> _mockLogger;
private CredentialService _service;
[TestInitialize]
public void Setup()
{
_mockContext = new Mock<IDbContext>();
_mockEncryption = new Mock<IEncryptionService>();
_mockLogger = new Mock<ILogger<CredentialService>>();
_service = new CredentialService(
_mockContext.Object,
_mockEncryption.Object,
_mockLogger.Object);
}
[TestCleanup]
public void Cleanup()
{
_service?.Dispose();
}
}
```
#### Test Methods Pattern
```csharp
[TestMethod]
public async Task GetCredentialAsync_WithValidName_ReturnsCredential()
{
// Arrange
var credentialName = "TestCredential";
var expectedCredential = new CredentialEntity { Name = credentialName };
_mockContext.Setup(x => x.Credentials.FirstOrDefaultAsync(
It.IsAny<Expression<Func<CredentialEntity, bool>>>(),
It.IsAny<CancellationToken>()))
.ReturnsAsync(expectedCredential);
// Act
var result = await _service.GetCredentialAsync(credentialName);
// Assert
Assert.IsNotNull(result);
Assert.AreEqual(credentialName, result.Name);
_mockContext.Verify(x => x.Credentials.FirstOrDefaultAsync(
It.IsAny<Expression<Func<CredentialEntity, bool>>>(),
It.IsAny<CancellationToken>()), Times.Once);
}
[TestMethod]
public async Task GetCredentialAsync_WithNullName_ThrowsArgumentNullException()
{
// Act & Assert
await Assert.ThrowsExceptionAsync<ArgumentNullException>(
() => _service.GetCredentialAsync(null));
}
```
### Test di Integrazione
#### Database Integration Tests
```csharp
[TestClass]
public class DatabaseIntegrationTests
{
private TestDatabase _testDb;
private IDatabaseManager _databaseManager;
[TestInitialize]
public async Task Setup()
{
_testDb = await TestDatabase.CreateAsync();
_databaseManager = new EFCoreDatabaseManager(_testDb.ConnectionString);
}
[TestCleanup]
public async Task Cleanup()
{
await _testDb.DisposeAsync();
}
[TestMethod]
public async Task ExecuteQueryAsync_WithValidQuery_ReturnsData()
{
// Arrange
await _testDb.SeedDataAsync();
var query = "SELECT * FROM TestTable WHERE Id = 1";
// Act
var result = await _databaseManager.ExecuteQueryAsync<TestEntity>(query);
// Assert
Assert.IsNotNull(result);
Assert.IsTrue(result.Any());
}
}
```
#### REST API Integration Tests
```csharp
[TestClass]
public class RestApiIntegrationTests
{
private TestServer _testServer;
private HttpClient _client;
[TestInitialize]
public void Setup()
{
var builder = WebApplication.CreateBuilder();
// Configure test server
_testServer = new TestServer(builder);
_client = _testServer.CreateClient();
}
[TestMethod]
public async Task GetData_WithValidCredentials_ReturnsSuccess()
{
// Arrange
var apiClient = new GenericRestServiceClient(_client);
// Act
var result = await apiClient.GetAsync<ApiResponse>("/api/data");
// Assert
Assert.IsNotNull(result);
Assert.IsTrue(result.Success);
}
}
```
### Test End-to-End
#### Playwright UI Tests
```csharp
[TestClass]
public class UITests
{
private IPlaywright _playwright;
private IBrowser _browser;
private IPage _page;
[TestInitialize]
public async Task Setup()
{
_playwright = await Playwright.CreateAsync();
_browser = await _playwright.Chromium.LaunchAsync();
_page = await _browser.NewPageAsync();
}
[TestMethod]
public async Task DataTransfer_CompleteWorkflow_Success()
{
// Navigate to application
await _page.GotoAsync("http://localhost:7550");
// Configure source connection
await _page.ClickAsync("#source-database-tab");
await _page.FillAsync("#connection-name", "TestConnection");
// Configure mapping
await _page.ClickAsync("#next-step");
// Execute transfer
await _page.ClickAsync("#execute-transfer");
// Verify results
var successMessage = await _page.WaitForSelectorAsync(".success-message");
Assert.IsNotNull(successMessage);
}
}
```
### Test Configuration
#### appsettings.Test.json
```json
{
"ConnectionStrings": {
"DefaultConnection": "Data Source=:memory:",
"TestDatabase": "Server=(localdb)\\mssqllocaldb;Database=DataCouplerTest;Trusted_Connection=true;"
},
"Logging": {
"LogLevel": {
"Default": "Debug",
"Microsoft": "Warning"
}
},
"TestSettings": {
"UseInMemoryDatabase": true,
"MockExternalServices": true
}
}
```
### Test Execution Scripts
#### PowerShell Test Runner
```powershell
# run-tests.ps1
param(
[string]$Configuration = "Debug",
[string]$Filter = "",
[switch]$Coverage
)
Write-Host "Running tests..." -ForegroundColor Green
if ($Coverage) {
dotnet test Data_Coupler.sln `
--configuration $Configuration `
--collect:"XPlat Code Coverage" `
--results-directory TestResults `
--filter $Filter
# Generate coverage report
reportgenerator `
-reports:"TestResults/*/coverage.cobertura.xml" `
-targetdir:"TestResults/CoverageReport" `
-reporttypes:Html
} else {
dotnet test Data_Coupler.sln `
--configuration $Configuration `
--filter $Filter
}
```
## 🔒 Security Considerations
### Gestione Credenziali
#### Crittografia dei Dati Sensibili
```csharp
public class EncryptionService : IEncryptionService
{
private readonly IDataProtector _protector;
public EncryptionService(IDataProtectionProvider provider)
{
_protector = provider.CreateProtector("DataCoupler.Credentials.v1");
}
public string Encrypt(string plaintext)
{
if (string.IsNullOrEmpty(plaintext))
return plaintext;
return _protector.Protect(plaintext);
}
public string Decrypt(string ciphertext)
{
if (string.IsNullOrEmpty(ciphertext))
return ciphertext;
try
{
return _protector.Unprotect(ciphertext);
}
catch (CryptographicException)
{
// Handle decryption failure
throw new SecurityException("Failed to decrypt data");
}
}
}
```
#### Configurazione Data Protection
```csharp
// Program.cs
builder.Services.AddDataProtection()
.PersistKeysToFileSystem(new DirectoryInfo(keyPath))
.SetApplicationName("DataCoupler")
.SetDefaultKeyLifetime(TimeSpan.FromDays(90));
```
### Validazione Input
#### SQL Injection Prevention
```csharp
// ✅ Parametrized Queries
public async Task<List<Entity>> GetDataAsync(string tableName, int id)
{
// Validate table name against whitelist
if (!IsValidTableName(tableName))
throw new ArgumentException("Invalid table name");
var query = $"SELECT * FROM [{tableName}] WHERE Id = @id";
var parameters = new { id };
return await _connection.QueryAsync<Entity>(query, parameters);
}
// ❌ Mai utilizzare string concatenation
public async Task<List<Entity>> GetDataUnsafe(string tableName, int id)
{
var query = $"SELECT * FROM {tableName} WHERE Id = {id}"; // VULNERABLE
return await _connection.QueryAsync<Entity>(query);
}
```
#### Input Sanitization
```csharp
public class InputValidator
{
private static readonly Regex AllowedTableName = new(@"^[a-zA-Z][a-zA-Z0-9_]*$");
private static readonly Regex AllowedFieldName = new(@"^[a-zA-Z][a-zA-Z0-9_]*$");
public static bool IsValidTableName(string tableName)
{
return !string.IsNullOrWhiteSpace(tableName) &&
tableName.Length <= 128 &&
AllowedTableName.IsMatch(tableName);
}
public static bool IsValidFieldName(string fieldName)
{
return !string.IsNullOrWhiteSpace(fieldName) &&
fieldName.Length <= 128 &&
AllowedFieldName.IsMatch(fieldName);
}
public static string SanitizeStringInput(string input, int maxLength = 255)
{
if (string.IsNullOrEmpty(input))
return string.Empty;
// Remove potential XSS characters
input = input.Replace("<", "&lt;")
.Replace(">", "&gt;")
.Replace("\"", "&quot;")
.Replace("'", "&#x27;")
.Replace("/", "&#x2F;");
return input.Length > maxLength ? input.Substring(0, maxLength) : input;
}
}
```
### Autenticazione e Autorizzazione
#### Connection Security
```csharp
public class SecureConnectionFactory : IDataConnectionFactory
{
public async Task<IDatabaseManager> CreateDatabaseManagerAsync(string credentialName)
{
var credential = await _credentialService.GetCredentialAsync(credentialName);
if (credential == null)
throw new UnauthorizedAccessException("Credential not found");
// Validate credential permissions
if (!await ValidateCredentialPermissions(credential))
throw new UnauthorizedAccessException("Insufficient permissions");
// Create connection with timeout
var connectionString = BuildSecureConnectionString(credential);
return new DatabaseManager(connectionString);
}
private string BuildSecureConnectionString(CredentialEntity credential)
{
var builder = new SqlConnectionStringBuilder
{
DataSource = credential.Host,
InitialCatalog = credential.DatabaseName,
UserID = credential.Username,
Password = _encryptionService.Decrypt(credential.EncryptedPassword),
// Security settings
Encrypt = true,
TrustServerCertificate = false,
ConnectTimeout = 30,
CommandTimeout = 300
};
return builder.ConnectionString;
}
}
```
### HTTPS e Comunicazioni Sicure
#### HTTPS Configuration
```csharp
// Program.cs
if (!app.Environment.IsDevelopment())
{
app.UseHsts();
app.UseHttpsRedirection();
}
// Force HTTPS in production
app.Use(async (context, next) =>
{
if (!context.Request.IsHttps && !app.Environment.IsDevelopment())
{
var httpsUrl = $"https://{context.Request.Host}{context.Request.Path}{context.Request.QueryString}";
context.Response.Redirect(httpsUrl, permanent: true);
return;
}
await next();
});
```
#### HTTP Client Security
```csharp
public class SecureHttpClientFactory
{
public HttpClient CreateSecureClient(RestCredentialSettings settings)
{
var handler = new HttpClientHandler();
if (!settings.IgnoreSslErrors)
{
handler.ServerCertificateCustomValidationCallback =
(sender, cert, chain, sslPolicyErrors) =>
{
// Validate certificate chain
return sslPolicyErrors == SslPolicyErrors.None;
};
}
var client = new HttpClient(handler)
{
Timeout = TimeSpan.FromSeconds(settings.TimeoutSeconds)
};
// Add security headers
client.DefaultRequestHeaders.Add("User-Agent", "DataCoupler/1.0");
client.DefaultRequestHeaders.Add("X-Requested-With", "XMLHttpRequest");
return client;
}
}
```
### Logging e Audit
#### Secure Logging
```csharp
public class SecurityAuditLogger
{
private readonly ILogger<SecurityAuditLogger> _logger;
public void LogCredentialAccess(string credentialName, string operation, string userId)
{
_logger.LogInformation(
"Credential Access: {Operation} on {CredentialName} by {UserId} at {Timestamp}",
operation, credentialName, userId, DateTime.UtcNow);
}
public void LogDataTransfer(string sourceType, string destinationType, int recordCount, string userId)
{
_logger.LogInformation(
"Data Transfer: {RecordCount} records from {SourceType} to {DestinationType} by {UserId} at {Timestamp}",
recordCount, sourceType, destinationType, userId, DateTime.UtcNow);
}
public void LogSecurityEvent(string eventType, string details, string userId)
{
_logger.LogWarning(
"Security Event: {EventType} - {Details} by {UserId} at {Timestamp}",
eventType, details, userId, DateTime.UtcNow);
}
}
```
### Error Handling Sicuro
#### Information Disclosure Prevention
```csharp
public class SecureErrorHandler
{
private readonly ILogger<SecureErrorHandler> _logger;
public ErrorResponse HandleError(Exception ex, bool isDevelopment)
{
// Log full error details
_logger.LogError(ex, "Error occurred: {Message}", ex.Message);
// Return sanitized error to client
if (isDevelopment)
{
return new ErrorResponse
{
Message = ex.Message,
StackTrace = ex.StackTrace,
Type = ex.GetType().Name
};
}
// Production: return generic message
return new ErrorResponse
{
Message = "An error occurred while processing your request",
ErrorId = Guid.NewGuid().ToString() // For correlation
};
}
}
```
### Database Security
#### Connection Security
```csharp
public class DatabaseSecuritySettings
{
public static SqlConnectionStringBuilder CreateSecureConnectionString(
string server, string database, string username, string password)
{
return new SqlConnectionStringBuilder
{
DataSource = server,
InitialCatalog = database,
UserID = username,
Password = password,
// Security settings
Encrypt = true,
TrustServerCertificate = false,
IntegratedSecurity = false,
MultipleActiveResultSets = false,
// Timeout settings
ConnectTimeout = 30,
CommandTimeout = 300,
// Connection pooling
Pooling = true,
MinPoolSize = 0,
MaxPoolSize = 100
};
}
}
```
---
**Versione**: 1.0
**Ultimo Aggiornamento**: Settembre 2025
**Framework**: .NET 9.0
**Sviluppatore**: Alessio Dalsanto